Value beyond scientific validity: let’s RULE (Reliability, Utility, LEgitimacy)

ABSTRACT

My perspective piece contributes to social studies of biometric technologies, and to studies on values and valuation within debates of responsible innovation. I reflect on innovation as social practice where values are temporary settlements of considerations around validity, operability, and social compatibility of socio-technical innovations. As such, I propose a practice-based approach to testing values in new technologies and their respective emerging practice and governance arrangements around Reliability, Utility and LEgitimacy (RULE). These three values combine scientific with operational and social aspects of innovation as centre-points around which deliberative engagement can be facilitated between different societal perspectives, offering the opportunity to develop greater awareness of diverse and at times competing understandings of value. On the case study of forensic genetics – the use of genetic material and data for policing purposes in security and justice contexts – I make the case for multi-perspectival, cross-disciplinary, community-grounded deliberation based on RULE.

KEYWORDS:

• Forensic genetics
• legitimacy
• reliability
• responsible innovation
• utility
• values

Innovation as social practice

In this perspective piece, I reflect on the social nature of innovation, and argue for a practice-informed approach to ethical deliberation that engages with concrete rather than abstract values. I propose an approach through which to facilitate dialogue about socio-technical innovation between different perspectives. This approach combines normative and practice-oriented considerations via three values of responsibility in innovation – Reliability, Utility, LEgitimacy (RULE) – which provide currency to the criteria of anticipation, inclusion, institutional reflexivity, and responsiveness (AIRR) (Stilgoe, Owen, and Macnaghten 2013) within the context of deliberative democracy.

In December 2016, German public debate about deployment of DNA technologies in policing re-ignited in the wake of the recent sexual murder of Maria Ladenburger in the city of Freiburg. Suspicion quickly fell on the community of one million refugees coming to Germany in 2015/2016, particularly from Syria, Afghanistan and Iraq. Large parts of the media, politics and interest groups played to an assumed link between migration and criminality: In what appeared to be an effort to shore up security from crime perpetrated by visibly different foreigners, the debate quickly revolved around the introduction into police use of emerging genetic analyses that may be able to infer appearance – primarily eye, hair and skin pigmentation – and ‘biogeographic ancestry’ (linking a person genetically to a population currently residing in a subcontinental geographic area) of unknown individuals, using biological traces found at scenes of crime (forensic DNA phenotyping).

Despite the lack of utility of forensic DNA phenotyping (from here: FDP) in this murder investigation – the police identified suspect Hussein Khavari using common policing methods – a vocal prominent group of law enforcement officers, political opinion- and decision-makers and scientists argued in favour of socio-technical innovation. They saw the deployment of FDP as legitimate even though it was not considered legal at the time, calling for a revision of the law to permit routine use in policing by ascribing significant value to them as law and order tools. The German Criminal Code of Conduct (Strafprozessordnung) had until then been interpreted to permit only sex determination and comparison of individual DNA profiles. FDP, however, may provide unique personal information about appearance, health, ancestry – based on probabilities derived from population reference data – of the trace donor (Kayser 2015; Phillips 2015). The reliability of such analyses; how they can best support police investigations; who should apply them; and who access their information, remain contested, reflecting a variety of competing values and interests (cf. Granja and Machado 2020; Samuel and Prainsack 2019; Wienroth 2018, 2020). While the German debate repeatedly referred to the permissive legal status for FDP in the United Kingdom and the Netherlands it took little note of context, use, and oversight there. This call for socio-technical innovation in the German criminal justice system focused on ‘Tagespolitik’ related to migration and crime management priorities while ignoring societal and institutional considerations around (non-)use of FDP in other countries. There was a limited reflection on the circumstances in which FDP can be useful in police investigations, or how reliable the data from such analyses are, with most advocates citing only scientific literature on ‘ideal-type’ accuracy predictions for eye, hair and skin pigmentation analysis. ¹

In December 2019, the Strafprozessordnung, a federal law, was amended to include the inference of appearance, ² but not the prediction of ‘biogeographic ancestry’, in part based on contestations of the legitimacy of a DNA-based notion of ‘race’ (cf. Lipphardt 2018). While the German state needed to be seen to be active in addressing criminality that has been associated with migration, its legislators were also aware of the history of race science and eugenics particularly in the twentieth century, and the cognitive error in associating genetic data, which seemingly fit into social categories, with naturalising such categories (the fallacy of biological racial populations). The legislature in the German state of Bavaria, however, introduced both analyses in the Bavarian Policing Act 2018, on the basis of their assumed value for building resilience to ‘imminent danger’. ³ Meanwhile, in the USA and elsewhere, private providers offer proprietary genetic analyses for appearance, ancestry, and familial kinship including genealogy (cf. Kennett 2019; Phillips 2018; Syndercombe Court 2018). The process of socio-technical innovation there has overtaken the capacity of existing governance structures to provide guided oversight in a heavily commercialised forensic market. At the same time, new technological innovations are underway in a field broadly described as ‘forensic (epi)genomics’ (Vidaki and Kayser 2017; Scudder et al. 2018), suggesting future capacity to infer health-relevant and life-style information from genetic material.

Initially, the German debate did not include a discussion of social context and innovations that necessarily go hand-in-hand with the introduction of new technological capacities into practice. And when it did, key decision-makers in Government and Police still deferred to scientific assurances of innovative potential (Lipphardt 2018). When it comes to socio-technical innovation in security and justice contexts globally, science, law and the state operate as core communities in decision-making. Recently, victim representatives ⁴ and commercial service providers ⁵ have acquired a stronger role in public debate on criminal justice innovations. However, these are extraordinary interventions. More can and needs to be done in terms of routine inclusivity of a broader palette of epistemic cultures and communities: a plurality of perspectives can and will better anticipate social convergences (cf. Wienroth and Rodrigues 2015) arising from technological innovations in practice (cf. Barben et al. 2008; Guston 2014). Within such a deliberative approach, we also need to do better in terms of providing a practicable basis for responsible deliberation that combines empirical with normative aspects (cf. Kon 2009).

The example of forensic genetics in criminal justice reflects on the social, even political nature of the ways in which socio-technical innovations are debated, legitimised and deployed. Look into how advanced genetic analyses are developed – e.g. how reference population data are created – and it becomes apparent that the basis for such innovations is not neutral: Scientific and technical considerations are subject to social communication and human nature, technological innovations are as much embedded within their societal context as they contribute to the shaping of social order and social practices. Sheila Jasanoff’s (2004) notion of the co-production of technology and social orders, and Lynch and McNally’s (2009) concept of biolegality – the interlinking of technology, practice and the social norms and legal frameworks governing both – have persuasively reflected on the situated social nature of socio-technical innovation. For forensic technologies, this context consists of scientific cultures and practices in research and case work; legal and regulatory frameworks; policing cultures and investigative practices; the market (forensic service provision and technology development); subjects of technology use (e.g. victims and their families, suspect individuals or communities); and public debates (e.g. catastrophising and criminalising migration). These constitute the deliberative horizon within which innovation is negotiated, materialised or closed down. They constitute the political economy of socio-technical innovation, the complex entanglements of diverse communities, discourses and practices, be they academic, lab-based, policing, commercial etc., that mark innovation as social practice (for FDP cf. Wienroth 2020). For the stage of adoption into material practice (cf. Tomlin et al. 2012), innovation has been described as ‘distributed effect’ (Sørensen 2007) based on its relational materiality in the entanglement of scientific, physical and social aspects: Innovation has material quality as well as object agency. The example of socio-technical innovation around enhanced forensic genetics analyses lends itself well to this point since it is steeped in DNA, chromosomes and alleles, mouth swabs, testing kits and analysis machines etc. as physical representations of suspicion, security and criminal justice. Socially, it is more widely distributed in policing and legal instruments as well as in the quality and nature of professional and public discourses about law and order, criminality, otherness, and migration that may lead to adoption, appropriation, and further innovation.

Enabling responsibility to RULE innovation

The German FDP debate shows how differing values play out in socio-technical innovations, illuminating them as social processes of valuing (cf. Helgesson and Muniesa 2013; Muniesa 2011; Boenink and Kudina 2020), and emphasising that deliberation of values of and in practice is vital in innovation processes. Extant RI scholarship provides structured procedural values for approaching intervention in such processes (Pellé 2016). Equally, through inclusive modes of engagement across various publics, policymakers and practitioners, responsible innovation makes space for the debate, production, and recognition of public values (Taebi et al. 2014). Such negotiation of values benefits from transparency and clarity in communication as well as perception of the issues at stake. However, communication can be challenging (Blok 2014), especially when values are not scrutinised, or if material, concrete elements of innovation are under-explored in favour of abstract concerns (e.g. the idea that any tool that may help in ‘catching a killer’ should be made available, irrespective of its scientific, operational, and societal limitations and impacts). Within this normative deliberative frame, I offer some granularity and accessibility in proposing three values – Reliability, Utility, LEgitimacy – that provide practical guidance for an anticipatory, inclusive, reflective and responsive innovation process. These values form a ‘golden’ RULE based on three dialogical questions asked of the technology, its potential adoption and distribution into practice, and its wider societal context:

1. Is the technology Reliable in delivering on expectations about its use?

2. Does it provide Utility to the task at hand (who defines ‘useful’ and ‘successful use’)?

3. Does its application/deployment satisfy questions of wider social LEgitimacy?

These questions – which help make complex innovation processes accessible to involved non-philosophical and non-academic communities – have emerged from my analytical work with social scientists, ethicists, civil society organisations, scientists, laboratory practitioners, and policy makers in forensic genetics, biometrics, and the life sciences more widely, including in the German debate on legislating FDP 2016–2019 (e.g. Wienroth and Lipphardt 2017).

RULE relies on inclusion, calling on stakeholder communities to participate in reflection and anticipation. The RULE values help open up discussion, enabling engagement because they start from an epistemic perspective, drawing from empirical and experiential – concrete – knowledge about specific socio-technical innovation, rather than abstract principles. Importantly, we need to be aware of the potentially very diverse aspects that make up reliability, utility, and legitimacy. These aspects must be critically discussed and constructively challenged to explore their links to specific issues, interests, and contexts within which they need to be considered. In this deliberative process, it is vital to draw on diverse forms of epistemic cultures, e.g. to address unconscious bias when discussing reliability and utility. For the example of forensic genetics, these cultures include forensic scientists and geneticists, police, constitutional and legal experts, political decision-makers, civil society organisations (as subjects to technology effects, e.g. minority organisations, victim groups), social commentators and ethicists, sociologists, anthropologists, criminologists, Science and Technology in Society (STS) scholars, as well as public and private forensic service providers. This is a long list of many expertises and insights, which it may be difficult to bring together and to get to communicate with each other, but considering the potentially severe impacts of socio-technical innovation on civil liberties, social order and life, potentially aggravating existing inequalities and inequities, such plurality lies at the heart of ethical and legitimate deliberation in liberal democracies.

Reliability

Reliability entails epistemic validity, veracity of data work, and user capacity. I argue that we need to understand these elements broadly as the thing in itself as well as those involved in producing/working/changing/interpreting the thing.

A key test for reliability lies in a technology’s scientific basis, its epistemic validity. ⁶ While validity in its narrowest sense is based on proof of principle, reliability also means achieving transparency about potentially inherent, and often invisible bias pertaining to data that informs technology. Bias derives from personal and cultural practices, prejudices, from social structures, ⁷ and from the inherently social – and political – nature of data of/about/by humans, also reflected in genetic research (cf. Lipphardt 2010; Saini 2019). As such, bias can also be incidental, unwanted, unrecognised by those subject to it. This offers another rationale for a plurality of views, experiences and expertise in innovation debates. Deliberation of epistemic validity of data, therefore, requires transparency about the context of, and structures that drive or enable socio-technical innovation. In the end, this is the baseline on which technology-enabled analysis will be performed and, by impacting on data created by technology-enabled analysis, inform social actions such as criminal justice investigations and trials.

Epistemic validity is closely related, therefore, to veracity of data work. With this type of veracity, I draw attention to the processes and agents involved in the production, curation, and use of databases and technologies as well as the analysis data informed and co-produced through such repositories and tools. What are their biases (in the same understanding as developed above, with emphasis on social context and structural embedding), are data workers aware of these, and how can such biases be mitigated? Therefore, veracity of data work refers to an understanding of how data are produced in technology-enabled enquiries and analyses. Veracity as a concept may have a positivist etymology, but for RULE purposes it is used to raise awareness to the need for procedural, not deterministic, testing of data work and data. As such, commitment to veracity here means establishing awareness of, as well as steps to identify and disclose, limitations of data technologies and their data. The informational content of data is an important aspect here (linking veracity closely to user capacity, see below), as is the question of compatibility of data processes and agents with the technical, and in turn social, capacity of the technology. A point raised time and again by oversight bodies here is that of confirmation bias: can users perceive the complexity of the social and often ambiguous data? In case of forensic DNA phenotyping, this can lead to misunderstanding the analysis and its probabilities (cf. Samuel and Prainsack 2019), potentially resulting in excluding specific appearance characteristics or population groups prematurely, or including others that later turn out to have little to no bearing on the investigation. These genetic attribution technologies and their related socio-legal innovations, e.g. in legislation and policing practice, have already shown to be subject to a variety of bias and misconceptions (see, e.g. Hansard Col 168W 2005; McCartney 2006; Phillips et al. 2009; Travis 2009; Tutton, Hauskeller, and Sturdy 2014), making the discussion of RULE explicitly important. Another element here is technology capacity: when it comes to technology deployment, potential users need to be able to understand if technology development has reached a readiness level at which analysis data can not only make statements relevant to the aim of its use (‘does it do what it says on the tin’), but at which error margins are sufficiently low. There needs to be consensus on how to manage unavoidable error margins, and what their limits look like. Equally, consensus is needed about the analytical framework. For example, what analysis and analysis tools, such as (open source) software, can be used; and how can data be interpreted reliably (‘what’s in the tin’)?

This has clear repercussions for user capacity, the use of socio-technical innovations in practice, such as in criminal investigations. Should data be used internally by the police, or can they be shared as part of a public appeal for information? If the latter, how can this be done in order to receive information from the public that can be used reliably and without endangering social order? Are data sufficiently unequivocal? In the end, reliability is always also a question of cross-checking information. In order to have user capacity, users need to be able to rely on disclosure of epistemic validity and have an understanding of the meaning of data veracity. This opens up questions of proprietary aspects of technology if offered as part of a commercial contract, e.g. in analysis software, outsourced data analysis, and generally in the delivery of routine and advanced forensic services (cf. Lawless and Williams 2010; Roberts 1996; Wienroth 2020). These are ongoing discussions, especially as commercial services providers grow in significance in some markets, e.g. the USA, and consolidate their range of services due to economic practices such as block-contracting and streamlined reporting requirements in others, e.g. the UK.

Utility

Utility considerations are closely linked to questions of reliability. The utility of an innovation depends on its operational value, e.g. if and how a technology and its analysis can add value to an investigation, and, ideally, includes strong consideration of how it can do so effectively. This needs to be accompanied by open discussion about the operational meaning of ‘effectiveness’ and ‘success’ of an intervention based on innovation. Scientific success in a case using forensic genetics technologies might not lead to investigative success, as shown in the example of the Phantom of Heilbronn (Lipphardt 2019). And effectiveness remains a difficult to measure concern of the criminal justice system (Amankwaa and McCartney 2019). Without clarity on these points, utility cannot be established satisfactorily.

More broadly, this point relates to questions of public/social order since technology deployment in the criminal justice system stands in service of (re-)establishing social order after a crime. Potential infringements of civil liberties and human dignity via technology deployment, however, can have a deleterious effect here if seen to be used without cause, or when used in inappropriate ways. Using forensic DNA phenotyping techniques to infer an unknown suspect’s potential ancestry and appearance, and sharing probability-based information without explanation publicly, can inadvertently fuel cultural prejudices, leading to ill will (or even violence) against minority groups. The analytical concept of ‘crimmigration’ in legal studies has taken stock of how immigration of some groups has increasingly been made a domain of criminal law (Garcia Hernandez 2013; Stumpf 2006), providing part of the cultural and structural context for the desire to innovate in order to better identify where a person comes from and how to protect against them. The German FDP debate provides a case in point of the merging of migration and criminal justice discourses and practices.

Equally, the considered use of such technologies, accompanied by appropriate training of criminal justice stakeholders on limitations and effects of technologies, as well as sensitive community relationships, can defuse critical challenges to public order arising from information shared, thus be useful to policing. One example is the early Marianne Vaatstra murder investigation when forensic phenotypical analysis suggested a white man as potential perpetrator in the Netherlands. This contributed to calming aggressions against a local community of asylum seekers who had initially been seen as implicated in the murder (cf. Jong and M’charek 2018). Here, investigators and citizens respected the limitations of the veracity of the data work in technology use. Therefore, the utility of socio-technical innovations, including the deployment of technologies, is closely linked to questions of reliability, and both values need to be considered carefully in tandem. The case can be made for their usefulness, but it stands to reason that, e.g. blanket deployment of specialist technology such as forensic DNA phenotyping does not generate operational value to an investigation. But it may cause damage to social order, such as civil unrest, violence, or social persecution based on an assumed identity and/or perception of race. In terms of informational utility, ⁸ mechanisms need to be established through which findings are communicated accessibly to science-lay users such as police detectives, prosecutors, lawyers, investigating magistrates, potential jurors (cf. Scudder et al. 2019).

Legitimacy

Legitimacy is about justification, trust, and social relationships. Legality, proportionality (an often imprecisely defined legal and political concept, cf. Urbina 2012) and other mostly technical terms are expressions of that value as its limited temporary settlements. Indeed, before a norm becomes law, its legitimacy is negotiated and contested at various levels. In these processes, the multi-perspectival deliberation of innovation is a vital tool in exploring legitimacy. Therefore, legitimacy of and in innovation is tested continuously and can be measured, e.g. in the relationship of technology deployment to ethical, moral, social, legal and other costs. Legitimacy testing needs to take place in the form of a broad discussion – between a diversity of perspectives from science, social science, ethics, politics, law, criminal justice, and civic society organisations including the voices of those often not heard – prior to the potential introduction of a technology for security and criminal justice purposes. In the context of legislation and/or regulation of technologies, and importantly also in the context of legal activities and legal restrictions, often new understandings of legitimacy are negotiated. Here, it is vital to attend to the political context and the political economy of debating the introduction of novel technologies: What has caused the call for change; which perspectives are heard, and which are given priority and why; are opportunities and successes of technology deployment, technical and operational limitations and challenges, as well as reliability and utility discussed equally? Ideally, the legitimacy of deploying technologies, which can lead to – at times justified – infringement of human dignity and civil liberties, should also be tested routinely in practice, e.g. by independent, preferably statutory, advisory bodies (such as the Biometrics Commissioners in England & Wales and Scotland). Important here is the reflection on existing – or lack thereof – legal and regulatory instruments that govern the deployment of technologies and, importantly, the retention of and access to data they produce.

Deliberative innovation is democratic

In attending to Reliability, Utility, and Legitimacy as key values in the deliberation of socio-technical innovations, we can actively incorporate values beyond those of technical capacity and economic worth when it comes to valuing science and technology, as part of responsible innovation. RULE supports accountability of innovation by increasing transparency while reducing the complexity of communicating about innovation by breaking down issues, and thus enhancing the capacity for inclusion of a plurality of perspectives in deliberation. Diversity in perspectives here is key, drawing on different experiences, knowledges and roles of stakeholder communities, be they involved in producing, using, consuming, governing or commissioning of, or being subject to, socio-technical innovation. Above and beyond that, we need to deepen and entrench consciousness of the social nature of technology development and deployment in considerations of responsibility. For example, Reliability, Utility, and Legitimacy as core values cannot be fulfilled by relying solely on scientific validation of a technology and its deployment in society. Instead, a more comprehensive approach is required to support the endeavour of providing equitable ‘public goods’ to a plurality of publics. While this paper does not offer the space to develop RULE into a comprehensive system of validation as valuation practice, constituted by epistemic, operational and societal validation (I am, however, working on this), RULE provides vital practical values for socially responsible innovation inclusive of multi-perspectival, cross-disciplinary engagement and debate in the public domain. Democratic deliberation of technological innovation requires well-informed, transparent and inclusive public debate, especially – as in the case of forensic genetics – when it comes to the use of scientific knowledge and socio-technological innovations that draw on the life sciences, and as such continuously co-produce social identity, the body as data, and life itself.

Acknowledgements

Many thanks to Robin Williams (Northumbria University) and Veronika Lipphardt (University of Freiburg) as well as many other colleagues for insightful discussions over the last few years which have led me to develop RULE. I would also like to thank the editors and the reviewer of this paper for their invaluable insight and comments on earlier versions.

Disclosure statement

No potential conflict of interest was reported by the author(s ).

Notes on contributor

Matthias Wienroth is Vice-Chancellor’s Senior Fellow at Northumbria University, and Visiting Fellow at Newcastle University, UK. His research focuses on science & technology as social practices, analysing knowledge production, (ethical, political, practical) value practices, and governance mechanisms in life sciences research and applications in the fields of biometrics, forensics, health. An integral part of his work is public, policy and practitioner engagement.

Notes

1 Further detail of critiques in the German debate can be found in Buchanan et al. (2018), Lipphardt (2017), Lipphardt (2018), Momsen (2017), Pfaffelhuber (2017), Ray (2018), Staubach (2017).

2 https://www.bmjv.de/SharedDocs/Gesetzgebungsverfahren/Dokumente/BGBl_Modernisierung_Strafverfahren.pdf (in German, accessed 8 July 2020).

3 Polizeiaufgabengesetz (PAG) Bayern, Art.32.4 (https://www.gesetze-bayern.de/Content/Document/BayPAG-32 in German, accessed 8 July 2020).

4 E.g. Karina Vetrano’s family advocating for familial searching in New York (2017), or Elizabeth Smart’s father – over a decade after her rescue from kidnapping – calling on the public to donate DNA to FamilyTreeDNA.com in televised adverts (2019).

5 E.g. see the aggressive marketing of forensic capabilities by Parabon Nanolabs in the USA.

6 When referring to epistemic validity I draw on Knorr Cetina’s (2007) ‘machineries of knowledge production’ to indicate the broader structural, cultural and societal aspects of data production/use/governance.

7 For example, structural racism disadvantages communities because of how the system works, not necessarily because of racist intent in individual human action.

8 Informational utility needs to consider the following questions: what can we learn from technology deployment; how can data be made useful; what are the differences between intelligence (something forensic DNA phenotyping can provide) and evidence (something phenotyping cannot) in terms of information and in terms of implications for investigative practices.