Does relevance matter in academic policy research?

This section of the journal encourages discussion between several authors on a policy-related topic. The same question may, therefore, be addressed from different theoretical, cultural or spatial perspectives. Dialogues may be applied or highly abstract. This Dialogue starts with this contribution and is followed by four comments by Huw Thomas http://dx.doi.org/10.1080/19407963.2014.990660; Lynn Minnaert http://dx.doi.org/10.1080/19407963.2014.990663; Noel Scott http://dx.doi.org/10.1080/19407963.2014.990665; Jan Mosedale http://dx.doi.org/10.1080/19407963.2014.990664 and, finally, Dianne Dredge's reflections prompted by the observations of fellow contributors http://dx.doi.org/10.1080/19407963.2014.990662

This is a deeply confronting question for most academic policy researchers, striking at the very heart of their value and contribution both inside and outside the academy. It is also a question that comes with baggage because there is an increasingly vocal section of the media that regularly questions the relevance of academic policy research and its lack of influence in policy decisions and actions. Research and its relevance is also linked to broader issues of the role of research in society and the notion of academic freedom, both of which have deep historical antecedents.

Inspired by Wilhelm von Humboldt's model for the University of Berlin (1810), most modern universities have been funded to undertake research and teaching that would contribute to a higher form of intellectual life from which society as a whole could benefit (Ferlie, Musselin, & Andresani, 2007). Under this model, academic freedom flourished for most of the nineteenth and twentieth centuries, with academics dedicating their lives to a teaching–research nexus that was considered the highest form of moral life (Prior McCarty, 2011). In the late twentieth century, the pressures of globalisation and neoliberal public management took hold. Governments began to focus on the effectiveness of public expenditure in higher education and policy shifts resulted in the increasing marketisation of educational products and contestability of public funds (Ayikoru, Tribe, & Airey, 2009). In research, measures of impact and significance have been employed as a means of promoting greater accountability in how research funds are used. In the process of aligning research activity with these performance measures, academic freedom, in terms of what to research and where to publish, has been significantly curbed. Moreover, university rankings based on narrow, overly simplified measures of research and teaching performance have emerged providing universities with the opportunity to extrapolate these performance outcomes into claims of excellence. As a result, strong performance in these indicators means more students are attracted and higher levels of funding can be achieved.

One important outcome of these changes is that the relevance of academic research has been framed in terms of how researchers’ and institutions’ outputs impact upon the field of study itself. The unfortunate result is that the impact and quality of policy research is now measured in terms of the quantity of published outputs in ranked journals and citation data (Hall, 2010). Universities incentivise researchers to publish in academic (not practitioner) journals and they are often dissuaded from time-consuming activities like engaging in policy discussions and knowledge transfer activities because they take time away from writing academic papers. The overall implication is that the relevance of research is now defined in terms of the bibliometric performance and not by the substantive issues that a publication addresses nor the way in which this knowledge is shared, discussed or how it influences the understandings and practices of those beyond the Ivory Tower (Jamal, Smith, & Watson, 2008). Not surprisingly, in many systems academic researchers are now ensconced in a system where competition and performance measures have a significant impact on the content and direction of their research (Marginson, 2007).

In this context, heckles from media or industry that academics are ‘irrelevant' (Kristoff, 2014), and that they possess a ‘mountain of exquisite knowledge surrounded by a vast moat of dreadful prose' (Lepore, 2013) are regular reminders that there are significant challenges in communicating the social value of academic policy research (Bastow, Dunleavy, & Tinkler, 2014). Others argue that academic research has failed to engage directly with policy-making, with research being ‘lost in translation  …  indigestible and obscure' (Shergold, 2011). I argue that this is not an issue of the relevance of academic research, but a more fundamental one about communicating the value of academic research. But before this argument can be unpacked, it is useful to explore two aspects that shape this debate.

What is academic policy research?

An inclusive definition of policy is adopted here that includes the nature of, influences on and outcomes from relationships between government, business and civil society that contribute to the act of governing (e.g. community engagement, partnerships, planning, policy-making, decision-making, implementation, evaluation, etc.). In the context of this definition, academic policy research includes a wide range of research including but not limited to the following:

• descriptive research – describes and analyses aspects of policy,

• normative research – provides policy guidance and recommendations,

• procedural research – describes and analyses planning and policy processes,

• predictive research – predicts the consequences of actions and strategies,

• evaluative research – evaluates the performance of policies and plans,

• relational research – assess relational characteristics, governance and partnerships,

• reflexive – accounts of how policy is made, experiential insights and influences.

It is not the aim here to evaluate the relevance of these different types of policy research, but rather to acknowledge that these approaches contribute different types of knowledge about policy. Moreover, it is important not to focus on the merits and shortfalls of particular types of policy research, for this can usually be traced back to the particular worldviews and methodological preferences of those who engage in such debates, but to appreciate the totality of the body of research and that the policy knowledge produced has different audiences, applications and makes different contributions.

What does ‘relevant’ mean?

Before we can come to any position on whether relevance matters, it is first important to examine what is ‘relevant research’. Those coming from the perspective of a single agency (e.g. international agency, tourism organisation, government department, firm, peak association, etc.) may have little trouble interpreting ‘relevant policy research’ in terms of the extent to which research informs the development of policy that furthers the missions, goals and objectives of that organisation. This instrumental view of relevance is narrow and fails to acknowledge a range of externalities (e.g. environmental impacts) and the management of public goods (e.g. natural resources) that policy must deal with. Academic policy researchers also hold diverse views about relevance. Those engaged in praxis and action research argue that it is possible to influence understandings and actions in the field through meaningful engagement with practice communities. Radical researchers tend to argue for more critical intellectual research that seeks to understand issues of power, the challenges of marginalised voices and silenced groups, the findings of which can be disseminated through education (Bramwell & Lane, 2006). In this radical view, policy change has a longer timeframe rather than the short-term focus of praxis researchers.

The issue of funding also comes into play in considering relevance. Policy research can be funded by diverse agencies with interests and influence over what is researched, the methods used and the audiences it targets. But we cannot say that funded research is more relevant than non-funded research. Funded research often reflects the realpolitik of the day and is shaped by political/power relations that do not necessarily reflect policy relevance or societal importance. Consulting research also reflects a trend towards maintaining existing industry-government arrangements and solutions that are acceptable to the funding agency. Put bluntly, policy-making is increasingly viewed as a game of mediated politics played between elected representatives and corporate interests and policy-makers. Policy research that seeks to open up new understandings and new approaches to complex protracted wicked policy problems such as climate change and sustainability are often sidelined (Bramwell & Lane, 2006). Confronting such issues is not a short-term endeavour: it requires commitment to academic independence and freedom, diversity of thinking, new ideas and critical reflection, which in turn create fertile conditions for innovative policy and practical policy measures.

This argumentation leads us to a post-structural position that there are multiple versions of what might constitute ‘relevant policy research’ depending on a range of factors including perspective, values, position, time and scale. Moreover, if we cannot define what is relevant academic policy research, then the question of whether or not it matters becomes redundant. Indeed, such a debate deflects attention away from the fact that a central tenet in policy research, even in its most theoretical and abstracted investigations, is an attempt to understand and theorise what is happening in processes of governing and policy-making and what can be done about it. The insights, reflections and learnings generated by adopting different lenses, theoretical frameworks and methodologies are necessary for understanding policy from different angles, and in maintaining a vital and creative space necessary for knowledge co-creation and policy innovation. Over time, research has helped to introduce new intellectual movements, alternative ways of understanding and the knowledge transmitted through academic policy research provides a smorgasbord of tools, frameworks, models and insights that can be regularly brought to bare on policy-making practice. Thus, academic policy research in a wide variety of forms (as long as it adheres to rigorous academic practice in terms of methods, ethical conduct and so on) provides a potentially rich landscape of knowledge that can help to interrogate policy problems in innovative ways, mobilise new understandings of persistent policy problems and uncover new policy actions. The real issue then is not one of relevance; it is how we can more effectively communicate the value of policy research – a challenge that makes it necessary for us to transcend our own situated knowledges and practices to understand a ‘bigger picture’. The responsibility for addressing this communication gap rests with a wide range of stakeholders inside and outside higher education including government, industry, universities, academics and media.

Relevance doesn't matter, communication does

In summary, the concept of relevance a dialectical concept: it is value-laden, problematic and it can be divisive. Defining what is relevant academic research is much like a dog chasing its own tail – a pointless pursuit aimed only at glimpsing ourselves. Such a focus only detracts attention away from the real issue which is communicating the value of academic research. But there are a number of serious barriers in converting this vast reservoir of knowledge into a malleable resource for different communities of practice (e.g. academic researchers, policy-makers, elected representatives, consultants, etc.). So how do we move forward? A tentative list of actions is offered as a way of addressing the communication gap:

• Reward public scholarship and engagement beyond the university by valuing it within academic work profiles, promotion and tenure processes.

• Acknowledge and support various kinds of praxis research and knowledge diffusion activities, including the delivery of training programmes by academics to practitioners, policy-makers and politicians; engaging in public debate and taking on advocacy roles; participating on advisory boards and stakeholder groups; and drafting and comment on public policy. These types of activities generally require more intensive, ongoing personal investment from researchers.

• Increase the access to and the operational proximities of researchers and practitioners, where there are opportunities to regularly and meaningfully engage in diverse forms of information exchange, knowledge co-creation and sense-making.

• Fund innovative and blue-sky research–practitioner collaborations that give social value to theoretically informed, creative and innovative research outcomes.

The above discussion also illustrates that this standpoint perspective (i.e. one that positions researchers and practitioners as discrete groups of sparring adversaries) is not helpful in finding a common platform for dialogue between researchers and the multiple stakeholders that could stand to benefit from diverse types of academic policy research. Good policy actions, both now and in the future, need both theoretical understanding and practical skills, the combination of which produces the practical wisdom or prudence to deal with complex real-world problems.

Disclosure statement

No potential conflict of interest was reported by the author.