Creating relevant knowledge in transdisciplinary research projects - Coping with inherent tensions

ABSTRACT

Transdisciplinarity aims to address ‘grand societal challenges’ especially in the sustainability area through realizing ‘alternative’ ways of knowledge production which integrate societal actors into research. Yet, despite considerable political and scholarly effort and support, the numerous initiatives and programs devoted to produce ‘societally relevant knowledge’ often yield rather conventional scientific output. For understanding why this is the case, this research investigates how researchers who were engaged in the Austrian research program proVISION made sense of ‘societal relevance’ and how they made it do-able in practice. The findings suggest rather uniform coping practices for dealing with inherent tensions between different kinds of relevance. Alignment between them could almost exclusively be achieved through quantification and computer modelling, which reinforced a positivistic understanding of relevance. The scarcity of more diverse collective coping strategies is taken as a starting point for developing conclusions for research funding.

KEYWORDS:

• Societal relevance
• socially robust knowledge
• sense-making
• do-ability
• transdisciplinary sustainability research

Relevant knowledge for societal challenges

For more than twenty years, it has been debated which kind of knowledge is needed to address the ‘grand challenges of our time’ (European Commission 2009). While scientific knowledge is considered to be key to understanding and addressing challenges like climate change or loss of natural resources, it has been claimed that alternative ways of knowledge production, which transgress disciplinary boundaries and open up research towards society, are essential to elaborating solutions (see, for example Brundtland 1987; United Nations Conference on Environment and Development 1992; European Commission 2012).

Yet, despite considerable policy efforts, observers doubt that there has been a broader transformation of knowledge production (Weingart 1997; Jahn, Bergmann, and Keil 2012), and empirical studies of inter- and transdisciplinary research find that the opening up of research towards society seems to remain limited to either the level of claims and aims, or to elaborating narrow problems (Hackett and Rhoten 2009; Woelert and Millar 2013; Polk 2014; Felt et al. 2016; Zscheischler, Rogga, and Busse 2017). Against this backdrop, I suggest that the best way to understand the barriers to realising all the well-designed and well-meant initiatives might not be found on a conceptual level, but rather in research practice itself and in paying attention to the ways researchers translate and realise policy concepts and funding requirements in practice and how they deal with tensions that they encounter (for similar approaches see Maasen and Lieven 2006; Bos et al. 2014; Felt et al. 2016; Glerup, Davies, and Horst 2017; Åm 2019).

I focus on transdisciplinary research (Hirsch Hadorn et al. 2008) as one approach whose central aim is to produce knowledge for addressing the ‘grand challenges’ through the close collaboration of scientists and societal actors. ¹ In particular, the policy discourse of the German-speaking countries refers to knowledge that is able to contribute to societal problem solving as being ‘societally relevant’. I take the Austrian transdisciplinary research program proVISION as a case study, as it explicitly aims to foster the production of societally relevant knowledge by integrating locally affected actors. At the same time, the program clearly aims to foster scientific research, and its assessment criteria include highly ranked publications. I thus use the proVISION program as a laboratory to explore how tensions between different demands are handled in practice and how researchers try to align ‘societal relevance’ and ‘scientific relevance’ within research projects.

The empirical findings are developed in three steps: I describe which tensions researchers who are engaged in the transdisciplinary projects of the proVISION program experience between policy relevance, practical relevance and scientific relevance in their research projects. I find that tensions between these different kinds of relevance occur in the context of three different dimensions: The relation of knowledge to reality, to values and to existing knowledge. I show how in the transdisciplinary projects alignment of the three kinds of relevance is achieved and how researchers cope with the tensions between them. I find that, in the proVISION projects, attempts to align policy relevance and practical relevance often did not work out well. The only satisfactory way to produce knowledge that is at once policy-relevant and scientifically relevant was to do so in the form of the ‘standardized package’ (Fujimura 1992) of computer modelling that is based on quantification and that excludes a range of other knowledge practices.

The contribution of the paper is twofold: First, I develop a conceptual contribution to STS literature on the process of how relevance is constituted. In doing so, I draw together approaches that focus on how researchers make problems do-able (Fujimura 1992) by aligning them with different cooperative settings and more recent approaches that take into account how researchers make sense of specific kinds of relevance (Felt 2009; Granjou and Arpin 2015; Klenk and Meehan 2017). Second, the empirical insights contribute to understand why despite political commitment, research funding and personal motivation of the involved researchers, initiatives for opening up knowledge production often yield rather conventional scientific outcomes. I argue that due to a lack of available collective coping strategies for dealing with inherent tensions between different kinds of relevance, researchers fall back to familiar and already accepted practices that are mostly based on quantification. This in turn reinforces positivist epistemologies. This insight is taken as starting point to draw conclusions for research funding.

In what follows, I elaborate on how societal relevance and the involvement of ‘affected actors’ are related in the literature on the concept of transdisciplinarity, and I discuss STS studies on inter- and transdisciplinary research practice.

Transdisciplinarity – Integrating ‘affected’ actors

While the purpose of public research funding has always been to provide benefits for society, there has been a profound change in what kinds of benefits policy actors expect and how policy governs scientific research. Since at least the end of the twentieth century, research governance has increasingly questioned the sufficiency of scientific self-governance, which is seen to be inwards-oriented, fragmented and narrow (Elzinga 2012). Thus, the relevance of research outcomes is no longer assessed only ex-post. Instead, funding proposals require that the potential societal benefits and how they should be reached must be argued in advance (Rip 1997; Demeritt 2000; Torka 2006; Hessels, van Lente, and Smits 2009). Moreover, more explicit steering and accountability measures have been put into place. Elzinga (2012) calls this the ‘period of accountability’ in science governance, which has brought about diverse ways of measuring knowledge production and its outcomes.

In policy and scholarly circles, it has been argued that mere accountability, in terms of addressing specific indicators, has adverse effects and that it does not encourage scientists to contribute to a ‘common good’ (Jasanoff 2003). Instead, creating closer relations between science and society is seen as being more likely to sensitize scientists for societal concerns and needs. In this respect, literature on transdisciplinarity as well as transdisciplinary initiatives and programs refer mostly to Nowotny’s (2003) call for ‘democratising expertise’ in policy-relevant areas by integrating societal actors not only into decision-making but also into knowledge production. This would make knowledge ‘socially robust’, which would be an addition to the ‘reliability’ of knowledge that ought to remain the basic condition of science. Nowotny argues that socially robust knowledge is continually ‘tested’ by the non-scientific actors within knowledge production. This in turn should ensure that the knowledge that is produced is relevant for these actors as well as for broader societal interests, which these actors are thought to represent. Thus, Nowotny conceptualises social robustness as a relational process that would keep scientists ‘aware of the societal context for their work’ (155).

While in the context of transdisciplinarity it is predominantly referred to ‘social robustness’, more recent debates on responsible research and innovation (RRI, Von Schomberg 2011) call for ‘societal responsiveness’ for creating links between societal demands, decision making and scientific research and innovation. While there are some overlaps between these concepts, responsiveness has a closer relation to policy processes and innovation governance. At least in its stronger versions (see, for instance Owen, Macnaghten, and Stilgoe 2012), it refers to an institutionalised ‘coupling of reflection and deliberation to action that has a material influence on the direction and trajectory of innovation itself.’ (Owen et al. 2013, 29) Thus, responsiveness calls for collective responsibility, which encompasses the whole ‘innovation ecosystem’ (46), while social robustness relies foremost on awareness and responsibility of individual researchers who cooperate with societal actors.

Literature on transdisciplinarity (which builds on Nowotny’s call for socially robust knowledge) explicitly refers to ‘societally relevant knowledge’ (gesellschaftlich relevantes Wissen) as knowledge that would help to address ‘grand challenges’ and their impacts through integrating actors from science and society in a joint research process (Hirsch Hadorn et al. 2008). Societally relevant knowledge should contribute to solving so-called ‘life-world problems’, which are already perceived to be problems prior to the scientific definition of the problem, and which extra-scientific actors attempt to solve but cannot do so alone (Hirsch Hadorn et al. 2008). Thus, there is a special emphasis on including so-called ‘affected’ actors who are locally bound to the problem to be solved.

Societal actors are taken to be representatives for society at large, and their problems are taken to be related to the ‘grand challenges’. The analysis at hand challenges this assumption. It shows that there are not only tensions between which knowledge is relevant within and outside of science; but there are also inherent tensions between knowledge that is relevant for specific local actors and their problems and knowledge that is relevant for society at large and long-term challenges. It further shows how attempts to cope with these tensions in transdisciplinary research practice in order to produce ‘socially robust’ knowledge enact societal relevance in specific and restricted ways.

In addition to the assumption that ‘affected actors’ would ensure ‘social robustness’, and thus relevance, another argument for this kind of problem solving is that those who are ‘affected’ by a problem possess specific local knowledge and experiences that are needed to develop solutions (Hirsch Hadorn et al. 2008). In this way, affected actors are at once conceptualised as contributing to and benefiting from transdisciplinary research.

Transdisciplinary knowledge in practice – Tensions and challenges

There is a body of literature that reflects upon transdisciplinary practices mostly in the own research projects with the purpose of facilitating the practical implementation of this approach (e.g. Gonzalo-Turpin, Couix, and Hazard 2008; Enengel et al. 2012; Lang et al. 2012; Simon and Schiemer 2015). These papers mostly deduct relevance criteria from the conceptual literature (see 1.1) and assess how well these criteria are met in the projects and what challenges they met. While these studies provide important insights into practical constraints for fulfilling transdisciplinary relevance criteria, they neglect the process of how researchers make sense of these criteria and how they translate them into practice. I thus focus on more recent studies from STS perspectives, which ask what knowledge the researchers who engage in transdisciplinarity themselves perceive to be relevant (e.g. Hackett and Rhoten 2009; Felt et al. 2013, 2016; Woelert and Millar 2013; Turner et al. 2015). They see relevance as ‘modes of mattering for one self and for others’ (Klenk and Meehan 2017, 28).

Furthermore, these studies examine what tensions researchers perceive between ‘societally relevant knowledge’ and other understandings of relevant knowledge. Tensions in transdisciplinary research are located on different levels (research groups, university programs or funding schemes), and they are described as ‘essential’ (Hackett 2005; Kuhn 1977) or paradoxical in the sense that they cannot be resolved once and for all, but that in research practice, researchers need to deal with them regularly. While such ‘essential tensions’ can also be found in disciplinary research collaborations (Hackett 2005), Turner et al. (2015) argue that tensions are experienced as more severe when collaboration goes beyond the own discipline (see also Andersen 2013). The analysis at hand complements this literature by developing a heuristic that shows that tensions between different kinds of relevance occur mostly along the relation of knowledge to reality, to values and to existing knowledge.

Most of these studies identify tensions, yet there are only a few that also analyse how researchers deal with such tensions in practice. Hackett and Rhoten (2009) analyse a transdisciplinary education program, and Felt et al. (2013) investigate a transdisciplinary doctoral program. Both find that the (mostly disciplinary) assessment criteria that decide researchers’ livelihood in the current science system prevail over societal relevance criteria, especially towards the end of such programs. Thus, students in these programs do not resolve the tensions they encounter, but gradually withdraw from inter- and transdisciplinary ways of working and try to strengthen their disciplinary profiles. While the students embrace the claim that science should produce societally relevant knowledge, they encounter difficulties in realising this claim within the current science system (see also Swan et al. 2010; Blättel-Mink and Kastenholz 2005).

There is a significant lack of empirical studies that explore how researchers cope with tensions between different kinds of relevance in transdisciplinary projects beyond teaching programs. Thus, looking at not only what kinds of knowledge researchers perceive as relevant, but also at the process of producing relevance seems expedient for exploring why claims and aims for alternative ways of knowledge production often fail to work out in practice. I address this gap by analysing the process through which researchers in transdisciplinary projects make sense (Blumer 1986; Berger and Luckmann [1966] 1991) of different kinds of relevance and how they align them in order to make the production of societally relevant knowledge do-able (Fujimura 1987).

Approach

Analytical approach – Making sense of relevance and making relevance do-able

As I analyse how societally relevant knowledge is made in research practice, I draw on STS approaches that deal with the process of creating relevance through a conceptual lens. These approaches do not take definitions of relevant knowledge for granted, but look at how knowledge comes to be accepted as relevant in specific moments and situations. Thus, they do not a priori distinguish between societal relevance and scientific relevance, rather they see such distinctions as the outcome of a translation process. From this perspective, relevance is a relational achievement (compare Klenk and Meehan 2017), a set of relations between actors that is constantly enacted.

The model of ‘translation’ (Callon 1986) focuses on how actors try to make their own knowledge relevant for others and thus to stabilise their own position within a network. In the process of ‘translation’, actor roles, interests and relations are negotiated simultaneously. This approach constitutes scientists as rather strategic actors who create relevance by convincing or mobilising others (for critique of such strategic models see Knorr-Cetina 1982; Fujimura 1992; Law 1999; Klenk and Meehan 2017).

Arguing that ‘translation’ foregrounds competition rather than cooperation, Fujimura (1987) develops an approach that addresses how specific lines of research are made ‘do-able’ by being aligned within different cooperative settings. For basic cancer research, Fujimura specifically names the three ‘levels of work organisation’ of the experiment, the laboratory and its resources, and the social world and its demands. She argues that only research problems that are able to align all three levels are do-able and thus can be realised.

A few more recent approaches have highlighted the tacit and emotional aspects of creating relevance, which also shape scientists’ understanding of relevance. For example, talking of ‘epistemic commitment’, Granjou and Arpin (2015) point to the fact that producing different kinds of relevant knowledge includes emotional, social and moral attachments. In addition, the concept of ‘epistemic living spaces’ (Felt 2009) pays attention to the tacit and emotional aspects of research practices and to individual and collective sense-making. What researchers perceive as relevant, desirable, possible and impossible demarcates their room for manoeuvre and also constitutes more collective environments. Klenk and Meehan (2017) deal with the relation of ‘meanings of relevance’ and researchers’ subject positions that develop as a relational achievement during the transdisciplinary research process.

I conceptualise the production of societally relevant knowledge in transdisciplinary research through the mutual relation of two processes: making sense of ‘relevant knowledge’ in epistemic, social and emotional ways (Felt 2009) and making the production of relevant knowledge do-able (Fujimura 1987) by aligning diverging commitments, concerns, requirements and practices. I thus do not take the interests or stakes of the researchers for granted, and I consider sense-making (for instance Blumer 1986; Czarniawska 2004; Berger and Luckmann [1966] 1991) as a crucial element of constituting relevant knowledge. I see sense-making and making do-able as being mutually related and situated. What seems do-able in a specific situation shapes sense-making, and what can be embedded into a specific situation in a meaningful way can be made do-able more easily.

Material and methods of analysis – The transdisciplinary research program proVISION as a laboratory

I draw on interviews, focus group discussions and observations of project meetings conducted in the project, ‘Transdisciplinarity as Culture and Practice’, ² which investigated the Austrian research program proVISION. proVISION includes many of the imaginations, demands and tensions around producing knowledge that is relevant to societal problems, which are applicable beyond Austria and beyond the specific concept of transdisciplinarity.

Overall, proVISION should create ‘the scientific basis for the country’s sustainability strategy’ (proVISION). ³ Accordingly, the program aimed to foster the production of policy-relevant knowledge that could be used for long-term provision and to build strategies for dealing with the impacts of phenomena such as climate change on a national level. The program takes an ambivalent position between different orientations of knowledge production. On the one hand, the program outline promotes an alternative way of doing research as a common process of scientists and affected actors. On the other hand, the assessment criteria support the current mainstream of performance-oriented research governance and build on scientific publications.

Thus, transdisciplinary research in the proVISION program can be regarded as something that requires an articulation of different demands and understandings of relevance. I regard the proVISION program as a laboratory for studying how researchers with different backgrounds and with different resources make sense of ‘societally relevant knowledge’ and how they make it do-able.

The data was collected within eleven of the projects that were funded by the proVISION program and in one doctoral school that was co-funded by the research program. It includes semi-structured interviews with 28 researchers, two focus groups and observation protocols of eight meetings held in three different projects. While the analysis for this article is based on all this material, the accounts of the eleven project leaders proved especially rich sources for analysing the articulation of different kinds of relevance. Project leaders are the ones who primarily need to understand and align different demands and needs, and thus can be regarded as mediators between the funding program and the different project participants.

The semi-structured interviews included (amongst others) questions concerning how the researchers characterise and assess the knowledge that is created in the concrete projects and in transdisciplinary research more generally, and how they see the relation between this and other kinds of (e.g. disciplinary) knowledge. Yet, even before we mentioned these questions, the researchers talked about which knowledge would be relevant to which actors, across all interviews. How to bring this together within the projects seemed to be a major concern for almost all of the interviewees and also in the project meetings that we observed. It was against this backdrop that my interest in the constitution of societally relevant knowledge in research practice first arose.

I developed a variation of Clarke’s (2005) ‘position mapping’ to analyse which understandings of ‘relevant knowledge’ can be found in the material. I identified different accounts of ‘knowledge that matters’ in the material through coding (Charmaz 2006). I then put these accounts on a map, and ordered them according to similarity and difference. Then, based on the empirical material, I added relations and demarcations made between different kinds of relevance. The mapping method helped me to avoid merely reproducing researchers’ accounts in favour of a more nuanced analysis of the process of constituting relevance. Mapping positions allowed me to see which shared stories researchers draw on in their sense-making (compare Czarniawska 2004). It also allowed me to analyse which positions are more easily articulated and related to each other, and which pose contradictions, providing hints to why certain understandings of relevant knowledge appear to be more easily do-able than others.

Producing societally relevant knowledge in transdisciplinary research

Making sense of tensions between different kinds of relevance

On first sight, it seems quite obvious what societal relevance means in the frame of the proVISION program: Whatever knowledge contributes to solving the broader sustainability problems that are the focus of the research program is relevant. Yet, when looking at researchers’ accounts and in their discussions in the project meetings, it is striking how the seemingly obvious meaning of societal relevance falls apart. In general, researchers acknowledge two different meanings of societal relevance: What I propose to call policy relevance, and what I propose to call practical relevance. In addition, the researchers emphasise that the knowledge produced in the transdisciplinary projects needs to be scientifically relevant. The researchers, especially the project leaders, do not only see specific actor groups as being in need of these different kinds of knowledge, but they also embrace all these relevance understandings themselves to different extents.

In the projects, they try to produce knowledge that would be accepted at once as policy-relevant, as practically relevant and as scientifically relevant. To do so, they try to come to grips with the different understandings of relevance and they try to make sense of their specificities and their differences. The analysis shows that the main differences between different kinds of relevance occur in the context of three dimensions: its relation to reality, to specific values and to existing knowledge (see Table 1).

Table 1. Different kinds of relevant knowledge.

	Relation to reality	Relation to values	Relation to existing knowledge
Policy-relevant knowledge	Knowledge that reflects the complexity of a situation that is made up of different interrelated known and unknown factors, to estimate the probable development of this situation when certain measures are taken	Knowledge that contributes to sustainability as a common good	Knowledge that bridges and complements existing bodies of legitimate knowledge to make them useful for decision making and regulation
Practically relevant knowledge	Knowledge that reflects the situated and immediately experienced threats of the local actors	Knowledge that contributes to secure local livelihood and identity	Knowledge that complements and relates existing knowledge and experiences in a practically useful way to create solutions
Scientifically relevant knowledge	Knowledge that reflects specific parts of reality in a valid way	Knowledge that reflects the value of objectivity and ‘scientific freedom’	New and innovative knowledge that goes beyond existing bodies of knowledge

Policy-relevant knowledge is often identified as the kind of knowledge that the proVISION program actually aims at. Many researchers describe the specific relation of policy-relevant knowledge to reality as reflecting the complexity of a situation that covers various interlinked influencing factors and how it would most likely develop in the future when certain measures are taken. Researchers often call this decision knowledge ⁴ . Policy-relevant knowledge in the proVISION program is understood to contribute to the value of sustainability that is framed as a ‘common good’. Policy-relevant knowledge bridges and complements existing bodies of knowledge to adapt and apply them to the specific problem and to facilitate decision-making and regulation.

Researchers in the proVISION projects understand practically relevant knowledge to be knowledge that the local practice actors demand. In the proVISION projects, local practice actors are, for example, farmers, local enterprises or tourism organisations. Researchers observe that knowledge that is relevant to the practice actors needs to relate to reality in a way that reflects situated and immediately experienced threats, such as how to react to agricultural cultivation problems, a decrease in revenues or altered weather conditions. Values that are addressed are local lifelihood and identity. This means that knowledge accepted as practically relevant needs to be able to increase, for example, the economic or strategic position of the practice actors, to strengthen what they are proud of. Practically relevant knowledge adds to existing practical knowledge and experience in ‘useful’ ways that create practical solutions. As examples of practically relevant project outputs, researchers mention regional marketing and training concepts or cultural initiatives that make it possible to adapt to environmental threats and that also strengthened the regional identity of the population.

Researchers also need to produce scientifically relevant knowledge (in most cases based on publications) in order to further and maintain their careers, but also to fulfil the assessment criteria of the proVISION program. To be publishable and thus scientifically relevant, knowledge needs to relate to reality in a valid way. Valid knowledge refers to an isolated aspect of a situation, or to causal relations between isolated factors. A core value of scientifically relevant knowledge that most researchers highlight, is objectivity, understood as neutrality towards specific interests and convictions. Researchers see scientifically relevant knowledge as going beyond existing knowledge, as being innovative and new.

Tensions between different kinds of relevance occur in relation to reality, values and existing knowledge.

Tensions around different relations of knowledge to reality

Here, the two main tensions appear around the question of how reality needs to be represented in order to deal with problems, and around problem definition.

The first tension around the representation of reality is mainly between policy relevance and scientific relevance. This tension is addressed, for example, in an interview with project leader PL1, ⁵ who has a natural science background, and who works in the field of environmental protection, at the intersection of policy and science. PL1 narrates in detail, over a lengthy interview section, that the complexity of land use decisions would comprise numerous factors, such as the material shape of the landscape, soil characteristics, micro climate and weather characteristics, traditions of cultivation practices in a certain region, laws and regulations, national and international market conditions, demand and competition, agricultural funding, and many more. He argues that the interplay of all these factors has been neglected so far, but that it would be necessary to depict reality adequately to build regulatory measures upon it. However, each of these factors could develop very differently depending on the composition of the other factors, and in many cases, it is not possible to predict how.

In practice these parameters naturally vary across a wide range. […] But the causal chain in between is far from unambiguous, there are probabilities in every step […] So, purely scientifically, one would have said before probably, well, that is so vague and so, […] one gets outcomes that are across such a spectrum that this is actually not scientifically interesting. Yet, it is exactly this spectrum that is actually interesting in practice. Because, if I am within an area of the spectrum that does not make sense any more, then I need to think about if I should take another way or try to optimise the whole system, or search for other solutions. (PL1)

In the interviews, researchers argue that in transdisciplinary research both comprehensiveness and validity need to be achieved. Thus, in project meetings they negotiate which elements can be made stable and how much complexity and uncertainty can be taken into account when a valid statement nonetheless must be made.

Tensions around the definition of the problem that should be elaborated in the projects appear mainly between policy relevance and practical relevance. Researchers recognise this tension between different problem understandings, for example, when practice actors deny being ‘affected’ by the policy problems that are defined in the projects. One PhD student who participated in the transdisciplinary doctoral school puts this in a nutshell:

And for them [the practice actors] it was just: Yes, why should we deal with that, if we do not even know that it will happen? Let’s rather wait until a problem comes up, until X happens, then we’ll find a solution. (DS1)

The discrepancy between affectedness as potential future exposure to changing conditions, and affectedness as experiencing harm from an already present condition is addressed by many researchers in the interviews. In the projects, they negotiate the range of societal problems.

Tensions around different relations of knowledge to values

Tensions appear mainly alongside the question of which role values should play in research, and alongside the question of whose values should be considered.

The first tension reflects a core conflict between the ideals of value-led and value-free research. It plays out mainly between policy relevance and scientific relevance. Almost all of the researchers who engaged in the proVISION program state that they personally embrace the idea that transdisciplinary knowledge would correspond with a common good and they strive to solve sustainability problems. Yet, many researchers express concerns that research that is explicitly value-led would be in tension with the crucial scientific value of objectivity. Consider the account of project leader PL2, who has a natural science background, works in an interdisciplinary institution and whose project deals with a complex ecosystem and its relation to social dynamics. PL2 narrates that balancing normative research motivations and an objective approach is indeed an ongoing concern for him and for his colleagues.

I already discussed with colleagues: how far does the scientist need to step back? So where … where it is necessary to objectify, and where one can also, indeed, bring in one’s position and say: yes, I stand for that and I would like to have it realised and for me this point is important and that’s where I want to go. (PL2)

While his personal motivation in the project was to contribute to the protection and sustainable management of a specific landscape, he argues that this normative motivation should not influence the scientific work on the project. Thus, through active self-control he tried to keep an objective stance in the project:

I also wanted something, and in this project my wants had to be turned off. Because as a scientist in this project, I actually want nothing, for the landscape. I did not want the [area] to become a beautiful [landscape type], an ecologically functioning [landscape type]. Naturally I want it as a person – that was the background motivation. (PL2)

Interestingly, PL2 explicitly emphasises that in this project he was not primarily interested in producing scientific knowledge, but rather in creating policy-relevant outcomes. His argument is that objectivity is an indispensable prerequisite for policy legitimacy, and thus for pursuing the normative goal of fostering sustainable development.

The second main tension concerning whose values should be considered is between the policy value of contributing to a common good and to particular practical values. While most researchers are sympathetic towards the need for practice actors to deal with practical threats, they are sceptical about how far such practical needs can actually be distinguished from the vested interests of specific actors. As a lecturer in the doctoral program notes: And trans has, well, this drawback, yes, it becomes contract research (L2). In the interviews, researchers erect a sharp boundary between a society-oriented transdisciplinary research (PL8) and what they label contract research, in the sense of a client who demands and defines the knowledge she needs for her individual interests. Although some researchers mention that it is also part of their work, apart from the proVISION program, to engage in contract research, they see it as illegitimate to use public funding for such research, which would not serve a broader common interest, but rather the particular interests of a specific actor. Thus, contract research is used almost as a swear word in the interviews and focus groups. Researchers formulate it as a danger to the policy relevance of transdisciplinary research and also as being in tension with scientific objectivity.

Tensions around different relations of knowledge to existing knowledge

To be relevant, knowledge needs to contribute to existing knowledge. Tensions occur mainly with respect to specific understandings of innovation and to the kinds of outputs that should be produced.

The tensions around different understandings of innovation occur between scientific relevance and both policy relevance and especially practical relevance. This is addressed, for example, by project leader PL2:

So, from a scientific perspective it is not genuinely thrilling if you say: okay, I want to get the most out of existing knowledge – but it was an important exercise for us, and I think also for the process, to simply aggregate the knowledge that is there for once. […] With this, one naturally doesn’t always scratch at the forefront of science – but we were aware of this from the beginning. (PL2)

While, in this way, most researchers who engage in the proVISION program acknowledge the value of supplemental knowledge for policy, they struggle much more with tensions between scientifically relevant knowledge and practically relevant knowledge. Researchers apprehend that unconventional or uncomfortable questions would not be posed if the practical demands of stakeholders needed to be addressed, as a lecturer in the transdisciplinary doctoral school remarks:

For me the question is crucial, how innovatively, how unconventionally can one work in the transdisciplinary format, so that is where I actually see the biggest limitation. Our projects are nice projects that, however, do not disrupt anything. I think, if one does research with a practical orientation, then, logically, one always stays within the realm of possibilities of these actor groups, that is the consequence, and totally unconventional ideas cannot arise as easily in this format. […] That excludes to a certain degree the pushing over of existing paradigms, so the super-innovative ideas —, well, what concerns really new thinking. (L1)

Here, the argument is that practice actors who have a certain stake in a problem would not be ready to support unpredictable and critical research that might not yield any immediately applicable solutions.

A related tension between practical relevance and scientific relevance that is especially pressing for researchers who need (or want) to further their career is that practically relevant knowledge that does not go beyond existing knowledge could not be published in scientific journals. Senior researchers who comment on the altered circumstances and pressures that younger researchers face nowadays also acknowledge that the production of knowledge that cannot be easily published is something that can only be afforded (R1) when a scientific reputation and position are already stabilised or when afterwards somewhere time is left (PL5). Thus, given limited time and limited resources, a trade-off between producing practical and scientific output needs to be made, and researchers need to carefully consider where to invest their energy.

A further tension regarding the kinds of output to be produced is between policy relevance and practical relevance. Researchers feel that knowledge produced in response to immediate practical problems would not be suitable for addressing broader and more long-term policy challenges, such as climate change. For example, instead of developing long-term strategy changes in agricultural land use or tourism, practice actors voiced more immediate aims within the projects, such as developing short-term marketing concepts or technical solutions. Thus, with regard to output, the focus of practically relevant knowledge on solutions to present problems is in tension with both scientific relevance and policy relevance.

Articulating policy relevance, practical relevance and scientific relevance

Overall, some projects seemed to struggle much more with addressing the tensions between policy relevance, practical relevance and scientific relevance than others. Those who could more easily manage to deal with the tensions almost exclusively applied computer modelling as a method. While this worked out well for those involved, it proves rather exclusive towards actors and kinds of knowledge that do not work with quantitative data. Further, both policy relevance and scientific relevance are in tension with practical relevance in ways that are often unresolvable.

Trying to align practical and policy relevance

Researchers experienced that a basic assumption of the transdisciplinarity concept could not be approved: that local actors who are involved in transdisciplinary projects would act as representatives of broader society and would thus secure that knowledge which is relevant for ‘grand challenges’ is produced. Rather, researchers problematized that the problems that affect the practice actors would often not be of broader societal relevance.

Researchers put some effort in trying to align the concrete problems of their practice partners with the ‘grand challenges’. For instance, during the projects, in workshops or information events, researchers tried to convince their societal partners that the broader problem that they elaborated indeed affected them. Some researchers framed this as raising the actors’ awareness for the problem, yet others talked of selling (DS2) them the problem. In addition, already when setting up their projects, researchers carefully selected societal project partners that would count as representing broader societal interests (see also Felt et al. 2016). Yet, for most projects it proved hard to find societal actors who were willing to participate at all, as PL8 explains:

The difficulty. Well on the one hand – one needs to note – to win stakeholders to turn to such processes. So it is really not so that one says: I have some topic and go into some region and pick up people then, yes? That is always the other way round, that they say: Look, I know someone there – OK? – they know the questions, they are probably obtainable for participating there – yes? […] And also within the processes it is not so, that then there is a representative group of stakeholders, but one, that somehow also shows up, yes? And no other, yes? But it is – how does that nice saying go? – those who are there are the right ones. And with those one then needs to cope with and … and deal with it and one … one probably also needs to reflect which statements are missing in this group, yes? So which ones are missing, which people?’ (PL8)

This quote shows that representativeness of their societal partners was indeed a crucial issue for the researchers. As they could not recruit a representative group of stakeholders, they tried at least to reflect upon how the group ‘that shows up’ would relate to the ideal of representativeness.

Further, as the researchers realised that the societal project partners actually did not profit from the transdisciplinary research that focussed on broader policy problems, and as they were not paid or compensated for their contributions either, they wanted to give something back (PL4). Many projects produced separate output for the practice actors. Yet, as time and resources in three-year-projects are scarce, the additional outcomes for the practice actors were often postponed and eventually neglected.

Articulating policy relevance and scientific relevance through modelling

In many projects, policy relevance and scientific relevance are articulated by creating computer models that can be used to calculate scenarios. This method is also proposed in the proVISION program’s calls for proposals. The computer models in the proVISION projects link different factors, such as climate, land use, biodiversity, pollution, resource supply and demand, the health of the population, economic developments, etc. The purpose is either to calculate their interrelated development over time or to calculate how specific factors develop when other factors change. In the projects, models are produced by defining causal links between the different factors in the form of input-output chains. The single factors are in turn elaborated by disciplinarily homogeneous teams.

To integrate knowledge into a computer model, it is necessary to make it ‘model-able’ (R4), as one researcher calls it. That means translating different kinds of knowledge into quantified information that can be combined and processed. To do so, researchers mainly use existing data sets (measurement data, survey data, economic measures, etc.) on the single factors or output-data from another sub-project and transform them into a common scale so that they can be combined. Knowledge that is not already quantified needs to be translated into quantitative information to be integrated into the model. For example, qualitative data, such as observation and interview data or historical records (for instance on the weather or on cultivation practices in a certain area) are coded and quantified. Local and experiential knowledge (memories, traditions, tacit, embodied knowledge and know-how) is made explicit and is for example translated into a quantitative factor that would weigh the probability of certain calculated land use scenarios. Of course, the transformation from qualitative knowledge into quantitative data does not go without losing its initial meaning to some extent (Porter 1995).

Once knowledge is cumulated in a common model that includes all the sub-parts, the different actors can in turn draw on the model’s output data and translate it back into their respective relevance frames. Quantitative data is treated as a kind of raw material (compare Edwards et al. 2011) – it can potentially be turned into publications for scientists, into decisions and legitimation for political actors, and even into practical solutions for local practitioners. Given that data is produced according to scientific standards, it is taken as a ‘non-cash benefit’ (PL1), as a commodity that can be converted into different currencies but which thereby keeps its overall value.

With respect to the tension between policy relevance as representing reality as a complex situation as a whole, and scientific relevance as representing isolated factors and their relations in a valid way, modelling makes it possible to include many factors and their relations and to calculate them in a valid way. Thus, complexity is addressed by including more aspects, and interactions between these aspects, into a model, without having to deal with the messy situation as a whole. To select the factors to be included in a model, projects often draw on the local experiential knowledge of their practice actors. In this way, they reduce complexity by excluding certain factors, but they ensure that they still deal with a ‘realistic’ model that reflects the locally experienced reality of the practice actors. Yet, just because the practice actors are included in the selection process, it does not necessarily mean that the outcomes of the modelling are practically relevant.

Modelling as a method also makes it possible to deal with the tension between value-led policy-relevant knowledge and objective scientifically relevant knowledge. The policy questions that should be addressed by the computer model are value-led, and they are about which measures are suitable to foster sustainable development. Yet, the model development itself, the linking of different data sets, can be framed as being value-free and objective. In a related way, modelling also provides a way to articulate different understandings of innovation within one project. Researchers produce what they understand to be new knowledge by focussing on methodological innovation, while the modelling produces aggregated data sets that combine and supplement existing knowledge in a way that makes it relevant for policy.

Modelling as equalising but exclusive

As a way of making the production of societally relevant knowledge do-able in research practice, modelling is equalising but exclusive (Igelsböck 2016). Scientific relevance criteria, like objectivity and validity, work as a common benchmark. Theodore Porter (1995) argues that objectivity, validity and quantification have historically developed in relation with policy making and administration (see also Scott 1998). Due to this common origin, in practice, it remains widely unproblematic to align policy relevance and scientific relevance by producing scientific models that address policy questions. Igelsböck (2016), who focuses on how models allocate roles and responsibilities to actors, captures the capability of modelling to draw together different actors and their knowledge with the notion of an integration machine. Yet, she argues that actors who do not work with models themselves and knowledge that cannot be easily quantified are excluded or marginalised. This is why Klenk and Meehan (2015) talk of integration as an ‘exclusionary practice’. It reproduces a rather ‘modern’ understanding of scientific knowledge as something that neutrally represents reality (Klenk and Meehan 2015).

Inclusion and exclusion happens mainly in two ways: First, models restrict what kind of knowledge is accepted as input (what can be quantified), and thus what gets included in the model or not. Second, the outcomes of modelling are only relevant to those actors who use quantitative data.

Modelling is inclusive as it not only integrates scientific data, but it also includes data produced by societal actors according to what is regarded as proper scientific procedures. For example, NGOs, public administration, citizen science initiatives, etc. can contribute their data to modelling. Yet, modelling is exclusive because it excludes or reduces embodied, tacit and interpretive knowledge. This concerns local practice knowledge as well as those social sciences and humanities that work with interpretive qualitative methods (on a marginalisation of social sciences and experiential knowledge in landuse-research see Zscheischler, Rogga, and Busse 2017). Such knowledge is only considered when selecting which factors or data sets should be included in a model, or when preparing the outcomes for dissemination. It is not seen as a genuine part of knowledge production. This means that only those aspects of realities that can be represented in a quantitative way are taken into account. As STS scholars have argued, this makes it possible to ask specific questions and develop specific solutions, but not others (Law 2004; Kleinman and Suryanarayanan 2012). Regarding the output of modelling, it works to articulate policy and scientific relevance, yet, it somehow leaves practical relevance aside. It requires extra work and additional calculations to develop practically relevant outcomes.

Discussion

This article started with the observation that even though new ways of knowledge production for securing societal relevance are expressed as a central aim in research policy and despite several initiatives’ and programs’ attempts to realise it, success seems to be limited (Woelert and Millar 2013; Polk 2014; Felt et al. 2016; Zscheischler, Rogga, and Busse 2017). The findings confirm that most projects that are funded in the frame of the transdisciplinary research program proVISION yield rather conventional outcomes – in the sense that the involved disciplines do hardly blur their methods and theories, and that societally relevant knowledge is almost exclusively achieved through applying quantification practices and modelling, which does not challenge established scientific relevance criteria. The limitation of the available coping strategies for reconciling different relevance understandings in relation to reality, values and existing knowledge are at least in part an explanation for this dynamic.

For instance, the tension between scientific validity and comprehensiveness of knowledge is mainly addressed through considering several aspects of a complex situation, but to elaborate them separately according to established scientific standards. As a result, the knowledge produced does not differ epistemically from disciplinary knowledge. While linking different aspects is of course valuable in different ways, it does not disrupt conventional ways of knowledge production. Likewise, the scientific value of objectivity is not questioned, but it is strengthened as a basis for both scientific and societal legitimacy. Epistemic innovation as a prerequisite for scientific publishing remains the primary goal.

Overall, most transdisciplinary projects do not challenge or change scientific relevance criteria, but they even reinforce a specific understanding of scientific relevance that is closely related to positivist epistemologies. Barriers to realise broader transformations of knowledge production can thus be found within the rather uniform strategies for aligning different understandings of relevance and in the lack of other collective strategies. This insight goes beyond literature that reflects upon challenges and constraints in the own transdisciplinary research (e.g. Gonzalo-Turpin, Couix, and Hazard 2008; Enengel et al. 2012; Simon and Schiemer 2015) and that mostly refers to methodological issues or insufficient personal or institutional commitment. Broader empirical studies on constraints for transdisciplinary sustainability research in Germany and Austria identify mostly cultural and structural challenges (Blättel-Mink and Kastenholz 2005; Felt et al. 2016; Zscheischler, Rogga, and Busse 2017). They argue that culturally entrenched disciplinary values, norms, practices and hierarchies are a main hindrance to transdisciplinary collaboration.

I argue here that a withdrawal to disciplinary values and practices might not only be the cause for but also a reaction to a lack of alternative collective coping strategies for dealing with inherent tensions. Swan et al. (2010) come to similar conclusions when studying a policy initiative for introducing a ‘Mode 2’ logic in the field of genetic sciences in the UK. They find that researchers react to the insecurities that go along with contradictions between the ‘old’ and the ‘new’ institutional logic by falling back to established practices that in turn reinforce traditional scientific modes. In a similar vein, Åm (2019) concludes that a lack of ‘meta-governance’ would explain why research policies translate only with difficulty into scientific practice.

In the case of the program that is analysed here, modelling seems to be almost the only collective coping strategy at hand to achieve alignment of scientific and societal relevance understandings. Here, modelling can be seen as what Fujimura (1992) calls a ‘standardized package’ that many researchers draw on to make the production of relevant knowledge in transdisciplinary projects do-able. The ‘standardized package’ of modelling combines certain methods and concepts in a way that can easily be linked to different social worlds. Fujimura argues that packaging enables both fact stabilisation (in this case: a certain causal relation between different factors) and coordination across social worlds. A standardized package is already accepted as credible and thus it makes dealing with tensions between different kinds of relevance relatively easy.

Other studies confirm that computer modelling can mediate between different domains and knowledges (Galison 1996; Knuuttila, Merz, and Mattila 2006; Igelsböck 2016) and modelling is acknowledged as a widely distributed method in environmental and sustainability research (Edwards et al. 2011; Granjou and Arpin 2015). Thus, the significance of modelling as a ‘standardized package’ for reconciling different kinds of relevance is most likely given beyond the research program that is investigated here. At the least, it can be assumed beyond this case, that there is a certain scarcity of accepted collective strategies beyond quantification for reconciling different kinds of relevance. This might explain, why more disruptive approaches do hardly appear, which works as a powerful barrier for realizing the transformative aspirations of many transdisciplinary initiatives.

Conclusions for research funding

The question arises how transdisciplinary research could be governed at the funding level to make it possible to realise also alternative understandings of societally relevant knowledge. Concluding from the insight that collective coping strategies are needed, research policy needs to actively seek to develop imaginations of how to align societal and scientific relevance beyond standardised quantitative tools, such as computer models. For example, Klenk and Meehan (2015) propose alternative approaches to transdisciplinary climate research that are not based on the ideal of integrating different kinds of knowledge according to positivist scientific criteria. Yet, it also needs to be considered how such less conventional ways of knowledge production could be reconciled with a scientific career (Hackett and Rhoten 2009; Felt et al. 2013, 2016; Turner et al. 2015). It needs to be clear that societally relevant knowledge is not necessarily associated with scientific relevance and with a high publication output (compare Nightingale and Scott 2007; Holbrook and Frodeman 2011; Bornmann 2012). To allow scientists to fully engage in the production of societally relevant knowledge, options for compensating the decreased publication output need to be developed. It is necessary to value transdisciplinary engagement apart from publications, for example by developing measures for ‘converting’ time spent in a transdisciplinary collaboration into a placeholder on a publication list that does not count as a negative in later career or funding decisions, and by adapting universities’ and funding institutions’ selection procedures accordingly.

Regarding the difficulty to align practical and policy relevance, questions that need to be clarified at the funding level are for example, if and which local practice actors should be contributors to knowledge production or beneficiaries. As contributors they must be adequately compensated for their contributions; as beneficiaries it needs to be decided what are legitimate demands and practice actors need to have a say in defining and evaluating these demands. In this sense, there is the question of whether specific local groups or what Callon (1999) calls ‘concerned groups’ should apply for funding and invite specific researchers or disciplines to contribute their knowledge to their practical problem, rather than the other way round.

In sum, policy needs to address both processes that are identified to contribute to the constitution of relevance here: sense-making by developing collective imaginations and do-ability by shaping institutional requirements and assessment practices. Without developing imaginations and possibilities of how policy relevance, practical relevance and scientific relevance could be reconciled on the project level and also within individual careers, it cannot be expected that broader transformations of knowledge production take place. Further, it is also irresponsible towards individual researchers to leave them alone with figuring out how to cope with inherent tensions. Against this backdrop, the attempt inherent in the concept of ‘socially robust knowledge’ (Nowotny 2003) to make researchers aware of societal actors’ needs through creating closer personal interactions can be observed in transdisciplinary projects. However, for realising the promises of a transformation of knowledge production also collective and action-oriented responsibility in the sense of ‘societal responsiveness’ (Owen et al. 2013) seems to be needed.

Acknowledgements

First of all, I owe thanks to the project team, Ulrike Felt (lead), Judith Igelsböck and Thomas Völker, who collaboratively produced the data this article is based on. I also want to thank Margaret Haderer and Max Fochler, the members of the Department of Science and Technology Studies at the University of Vienna as well as members of the endowed Chair for Sociology of Science at the Technical University Munich (TUM), who provided detailed feedback on earlier versions of this article. I am very grateful to the two anonymous reviewers and the editor, who engaged critically and productively with my article and thus helped to enhance it considerably. Not least, I thank all the researchers, practitioners and program representatives of the proVISION program, who gave us their time and shared their experiences and reflexions with us.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Andrea Schikowitz currently holds a postdoc position at the Friedrich Schiedel endowed chair for Sociology of Science at the Technical University of Munich (TUM). She works on collaborative knowledge practices in the creation of urban space and in ‘living labs’. She conducted her PhD research at the Department of Science and Technology Studies (STS) at the University of Vienna where she collaborated in the project ‘Transdisciplinarity as Culture and Practice’ until 2013. This article is based on her PhD thesis on changing practices of togetherness in research.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was supported by the Austrian Federal Ministry for Science and Research under the funding scheme proVISION (funding of the project ‘Transdisciplinarity as Culture and Practice’); and by the Dr. Maria Schaumayer scholarship for returners of the Vienna University of Economics and Business (amongst others, funding for publication projects).

Notes

1 Earlier definitions of ‘transdisciplinarity’ do not refer to joint problem solving but rather to a unitary scientific approach based on metaphysics and quantum physics that goes beyond disciplinary specifics (for an overview see Klein 2014). In this paper, I do not address this meaning of transdisciplinarity.

2 The material was produced collectively in the project ‘Transdisciplinarity as Culture and Practice’ (funded by the Austrian federal ministry for science and research) that was conducted at the Vienna Institute for Science and Technology Studies from 2009 to 2013. The project leader was Ulrike Felt, and project collaborators were Judith Igelsböck, Andrea Schikowitz and Thomas Völker. See https://sts.univie.ac.at/en/research/completed-research-projects/transdisciplinarity-as-culture-and-practice (accessed January 17, 2019).

3 Quotes from the research program are taken from its original website, the two calls for papers and the evaluation guidelines. As the text from these sources widely overlaps, quotes only refer to ‘proVISION’. The original website, which also published the mentioned documents, was online until 2014 at http://www.provision-research.at (accessed May 12, 2013). Due to the discontinuation of the program and a reorganisation of the responsibilities of the Austrian Ministries, the website was relocated in a shorter version to https://bmbwf.gv.at/forschung/national/programme-schwerpunkte/provision (accessed January 17, 2019).

4 If not indicated otherwise, the following quotes are taken from interviews with researchers engaged in the projects of the proVISION program. The original quotes are in German and translated by the author.

5 The interview quotes are coded in the following way: PL - ‘project leader’, R - ‘researcher’ (project collaborator), L - ‘lecturer’ (in the doctoral school) and DS - (student in the) ‘doctoral school’.