Understanding, capturing, and assessing value in collaborative design research

1. Introduction

The collection of papers in this special issue respond to a tension between the broadening scope, scale and heterogeneity of collaborative design research, and an ever-growing demand from funders and regulatory organisations for the measurement and benchmarking of the work researchers do (Wilsdon 2016). Participatory design and co-design practice produce value in diverse forms, over multiple timescales, captured by different groups against different agendas. In this special issue, we seek to explore how value of, and in, participatory and co-design practices could be understood and captured not only to serve the needs of research assessment – where the mechanisms and regimes of research impact and value tend towards highly defined, contained conceptions of value – but also to enhance the scale, scope, and resourcing of collaborative design. To this end, the authors collected here offer perspectives on emerging ethical and political engagements with ‘value’ as a contested terrain. In differing ways, they offer accounts of how we guide, and are guided by, practices of valuation in the complexities of collaborative design situations.

The motivation behind this SI emerges in part from our first-hand experiences working across large-scale collaborative design initiatives, in particular the ‘Knowledge Exchange Hubs for the Creative Economy’ (KE Hubs) programme funded by the UK Arts and Humanities Research Council (AHRC). This multi-million pound fund ran from 2012 to 2016, resourcing 4 regional hubs and 10 smaller initiatives, linking together research institutions across the UK (AHRC & Holden 2017). The explicit rationale of the scheme was to improve the ways in which universities worked with the creative sector, arguably under the auspices of the dissemination of research practice and insight – understood as ‘knowledge exchange’ – but with a strong backbone of demonstrable economic impacts. The outcomes of these ‘knowledge exchange’ activities were ultimately expressed in a range of interconnected metrics that included measures of engagement, participation and reach, job or company creation, new business models, changes in turnover, and the acquisition of both private and academic further funding. The accountability such measures move towards sits alongside the rise of the ‘third mission’ of universities seeking to create demonstrable value for academic research outside of the academy, particularly in social and economic contexts (Shore and McLauchlan 2012). For the KE Hubs an unstable political environment in the UK which included the Scottish independence referendum, a general election, and the build-up to Brexit, conspired to place both funder and funded projects under pressure to generate a very specific form of evidence of that activity, skewed towards narrow economic measures (Moreton 2016). The final reports of these hubs (see Dovey, Moreton, and Hargreaves 2016; Cooper 2015; Follett and Marra 2016; Senior 2016) were at pains to present the widest possible view of the hubs’ impacts, and the value they had generated for their partners and participants, yet they also demonstrated how the steer towards a narrowly defined regime of measurement made it hard for the distinct and competing value(s) produced to sit in dialogue with one another.

What we sought to do through our work in the KE Hubs, and now in this special issue, is to promote a conversation about how alternative ways of understanding value can be foregrounded in collaborative design research practice. The tension we have experienced through our work can be understood as a consequence of calculative regimes which construct, rather than reflect, the landscape that they describe to legitimise a particular form of governance or rule (Miller 2001). Our intent is instead to consider how value – broadly understood – can be reclaimed from this normative domain.

Other conditions and metrics for capturing value have been proposed that better reflect the social, cultural and environmental consequences of a range of activities, which often focus on experience and process, rather than outputs. Upton, Vallance, and Goddard (2014) criticise evaluations focussed on outcomes (products, publications, etc.) as being unsuitable for encouraging ‘knowledge exchange’, and instead propose an evaluation of the process. Similarly, Crossick and Kaszynska (2016) posit that the value of arts and culture is better understood in terms of personal affect (e.g. developing reflective, innovative and creative individuals, and engaged citizens), suggesting alternative ways to do formative and participatory evaluation through understanding creative processes.

Collaborative design research and practice is already engaged with the topic of value. Related research has reinforced the importance of cultivating and attending to the values of design participants and other stakeholders in collaborative design work (Iverson et al. 2012), developed practical methods and resources for doing so (Friedman, Hendry, and Borning 2017), and investigated the particular value of designers’ creative practice within collaborative work (Bowen et al. 2016). As the foci of design has broadened from artefacts to services, conceptualisations of value from other disciplines such as Consumer Culture Theory and Service Dominant Logic (Arvidsson 2006; Ng and Smith 2012; Vargo, Maglio, and Akaka 2008) are being utilised in developing design practices that understand how value is co-produced. Such work considers value within the frame of design practice – how value is understood, generated, and employed in design situations. Our special issue seeks to develop these practice-focussed perspectives into alternative ways of understanding value within the frame of design research assessment.

The goal of this special issue is to offer breadth, precision and practical utility in understanding and articulating how commensurable value can be exchanged as it passes beyond the boundaries and situations of collaborative design research practice. We have not sought (and nor have our contributing authors) to define, capture, or in anyway ‘fix’ value as a stable concept, but instead to seek ways in which conceptions of value and surrounding practices might enhance and strengthen collaborations. In doing so, we propose examining practices of valuation – the situated understandings and activities that serve to make things valuable to individuals and groups (Helgesson and Muniesa 2013; Kornberger et al. 2015). This perspective acknowledges the distinct values and forms of value that different individuals and groups will hold, and places the emphasis on the practices that could commensurate or separate them (Espeland and Stevens 1998) as well as being attendant to the conditions under which measurement or value capture takes place. This pragmatic frame offers resources for the researcher seeking to engage meaningfully with the multiple, contingent conceptions of value required to sustain and diversify collaborative design.

2. Collaborative design research and value ‘in practice’

We understand collaborative design research practice as seeking new ways of connecting people to shared and individual futures, unlocking, amplifying and catalysing individual creative potential, and contributing to broader, systematic shifts in governance, politics, and social practice. This expansion of design intent away from its well-defined and well-equipped pathways of products and projects, service provision, briefs and stakeholders, criteria and critique offers a new landscape of possibilities for the designer. This work is frequently underpinned by a recognition of the complexity and diversity of challenges that the world faces, the entanglements of knowledge and technology needed to address such challenges, and the heterogeneous perspectives of individuals implicated in them.

Consequently, uncovering, let alone understanding the diffuse, unpredictable and potential things understood to be ‘of value’ that emerge in such collaborative work, and to whom the value is ascribed, remains a significant challenge. A substantial body of work has developed which considers this challenge in the collaborative design context, offering new theoretical resources for researchers. Ontologies of value, such as chains and constellations have been appropriated (Normann and Ramirez 1993; Speed and Maxwell 2015); new ontologies of impact in infrastructures and infrastructuring have been deployed (Björgvinsson 2008; Hillgren, Seravalli & Emilson 2011); new conceptions of the relationship designers and design participants hold during and after the work of the designer ends have been proposed (Redström 2008; Bjögvinsson, Ehn, and Hillgren 2012; Bannon & Ehn 2012). These concepts help to describe the effects of collaborative design initiatives, and the often unpredictable, sometimes messy outcomes that they lead to. The interactions and assemblages that concepts such as infrastructuring seek to articulate have meaning beyond the existing impact of research; they point to potentials and actualities for exchanging ideas and social currencies to sustain collaborative design research initiatives.

This special issue recognises an acute need to consider how collaborative design researchers reflexively understand and capture the value that their practice and outputs offer to the world, and how their work addresses the demand for articulating quantifiable value without losing the distinctive theoretical and practical resources that it has to offer society. We must also consider how we as researchers embody, interrupt, challenge, or make problematic the kinds of value we seek to produce and capture. In preparing the call for this special issue, we thus explicitly sought applied work that engaged with questions about how collaborative design practices engage in the process of constituting, articulating and exchanging value, and how they negotiate the questions of power and subjectivity that come not only with having to measure and report on those values, but also with being complicit in the shaping and articulation of value in the first place. This ambition gave rise to a number of questions:

• By what political compunction are we being required to identify and articulate the contribution we make?

• How do existing valuation methods and metrics influence what we as researchers can see?

• Where might we uncover the intersections between the distinct practicalities of value that can connect together the interests of communities, assessors, and collaborators?

• How do we conceptualise the impacts of collaborative design research as both the work and its outcomes become distributed and diffused across the people, systems, practices, and places?

• How could we develop and perform valuation mechanisms that bridge the interests of researchers and those who resource their work?

The papers in this special issue tackle these broad, overlapping and sometimes contradictory questions from different perspectives. The authors’ accounts reflect the trickiness, slipperiness, and densely interwoven nature of value, valuation, power, politics, practice, and reflexivity.

The question of valuation – who is asking us to measure whom, to what end, and how – is picked up by Kimbell and Julier in their account of social design practice in the context of UK measurement and evaluative regimes. They unpack how audit culture, bureaucracy and management practices represent a space of incoherent and competing demands for evaluation, and that ‘assessing the “value” of social design research requires one to be aware of the devices, practices and infrastructures through which valorisation is practically achieved’. Their response is to use an imagined case study to offer a framework for conducting social design projects, as a mean to show how embedded in institutional and bureaucratic infrastructures evaluative regimes are.

Contributions from Robbins and Giaccardi, and Gray and Kou both tackle the question of what valuation practices in design research look like from different perspectives, drawing on empirical data from design practice. Robbins and Giaccardi offer two case studies of collaborative design research extending over several years, where design anthropologist Robbins collaborated with Internet of Things (IoT) designers in industry. Their account demonstrates the way in which values ‘get lost’ or are hard to capture ethically in collaborative design research practice. The authors tease out the question of causality and emergence, reflecting on how value generation at a programmatic level involves a decision, between taking credit for changes in practice made by participants that may well only tangentially be related to the intervention of their design research project.

Gray and Kou take a different approach, analysing discourse within an online design practitioner community. Their focus is on the practical valuing an electronically mediated community engages in as they share, audition, and collectively curate new pieces of design knowledge. Here the practice-connected nature of the research shows the potential of examining value negotiation within a community, rather than from the perspective of an external assessor. The discourse analysis methods used in the work also suggests the empirical possibilities in this domain with the availability of digital interaction records.

Luján Escalente extends critiques of institutional valuation mechanisms and focuses upon their apparent failure to capture the more complex aspects of participatory design research. She introduces a philosophical reading of value that acknowledges the ‘intra-actions’ across which value becomes a matter of agency and power; what it is to measure, produce, or imagine ‘value’ from the perspective of a philosophical reading of how embodied practice generates meaning. She challenges us to become more aware of how the production of ‘value’ or ‘valuing’ practices is embodied and situated in our research work, and practice.

Finally, this challenge of ethics, power, and responsibility is picked up by Agid and Chin. Their paper demonstrates how, on the ground, a form of reflexivity – concomitant with, but not reducible to – that offered by Luján Escalente, operates in contrasting projects in Haiti and New York. Their argument, in true interdisciplinary fashion, mobilises readings of feminist theory, postcolonial political economy, and development studies, to develop a perspective that values activity as it departs from the designer’s outputs and intent. They emphasise the need for multiple approaches that inform us as practitioners, researchers, and people wishing to make change.

3. Prompting debate and action

Seeking multiple perspectives on what is of value – and how things are made valuable – is the aim of this special issue. We have taken the position that the mechanisms that make work valuable emerge through practices, processes, and discourse, rather than from clear and certain answers. In this sense, the following contributions provide theoretical and practical discourse on the challenges and the rewards of foregrounding value in collaborative design research. They will provide the reader with new perspectives on this matter, and on the potential for future research to engage with the tension between existing conceptions and regimes of measuring value and the decentralised, distributed, and diffuse outcomes of truly collaborative activity.

This is, however, only a first step. The process of creating this special issue has thrown into relief the many rhetorical, theoretical, and practical functions that the term ‘value’ connects with in the collaborative design research community. We are not seeking an overarching definition or consensus on the term. Rather, we aim to enliven and activate a debate about this complexity that we feel will help us better address the challenges and opportunities surrounding value in our work. We suggest that this debate needs to continue, drawing in new theoretical and empirical research to create a safe space to examine the complex, contentious topics of value and valuation in collaborative design research.

As we investigate new opportunities to value our work, we must attend to the threats that evaluative regimes and mechanisms may introduce; threats to research work and to the collaborative potential our research explores. The importance of evaluating social and economic impact is recognised (European Commission 2010; Go8 & ATN 2012), but this is evaluative intent is often frustrated from meaningfully reaching beyond research outputs (Stern 2016) into contexts of impact beyond the academy (Watermeyer and Hedgecoe 2016; Hill 2016); it is fair to say that research assessment is only a single valuation perspective that bears on collaborative design research practice, but it is an increasingly important one.

Here we suggest that constructive approaches are warranted, proactively seeking to develop and embed the valuation practices that will meaningfully commensurate the needs of assessors with those of our collaborators. No such approach could satisfactorily represent the ‘true value’ of collaborative design activities, but it would give researchers and those with whom they collaborate a critical stake in the valuation processes that attempt to represent their work. Not only is design well equipped for such constructive work, but we suggest that collaborative design research in particular has the means and the motivation to connect with practices and concerns of those with whom we collaborate. This conceptual and practical work is already underway and we, the collaborative design research community, must build and maintain a strong role in it, or find the value of our work understood more through the eyes of others than through our own.

Additional information

Funding

This work was supported by the Arts and Humanities Research Council [AH/J005126/1,AH/J005150/1,AH/J005185/1].