Scaling up research on family justice using large-scale administrative data: an invitation to the socio-legal community

ABSTRACT

This article outlines the value of administrative data for family justice research. Although socio-legal scholars have extended their research beyond purely theoretical or doctrinal analyses, studies using large-scale digital datasets remain few in number. As new opportunities arise to link large-scale administrative datasets across health, education, welfare and justice, it is vital that the community of family justice researchers and analysts are supported to deliver research based on entire service or family court populations. In this context, this article provides a definition of administrative data, before outlining the potential of single, linked or blended administrative data sets for family justice research. The remaining sections of the article speak to questions that are pertinent to this particular academic community, including the distinctive contribution of the socio-legal scholar to interdisciplinary teams and the place of data providers in collaborative research. Drawing on the sociological concept of ‘publics’, the final section considers the multiple interest groups whose social licence must be secured, when personal records are used to understand the relationship between law and family life.

KEYWORDS:

• Administrative data
• justice
• socio-legal
• theory
• digital records

Introduction

There is a general consensus that the past two decades have witnessed a growth in empirical legal research, not least in response to sustained demand from policy-makers and practitioners (Cane and Kritzer 2010, Ho and Kramer 2013, Leeuw 2015, Creutzfeldt et al. 2019, Hamann 2019). However, there is also evidence of a longstanding empirical legal skills deficit internationally, due to limited research education for law students within undergraduate programmes (Genn et al. 2006). Arguably, academics working in the field of family justice, have been the most willing and able to conduct empirical work (Eekelaar and Maclean 1990, Huntington 2018). However, as yet, even family justice scholars have been slower to embrace new research opportunities afforded by the increasing digitisation of information within public institutions and the courts, when compared to colleagues in health research. Digital administrative records that are machine readable, together with opportunities to link large-scale administrative datasets across health, education, welfare and justice, enable vital insights to be gained based on entire service populations. As governments nationally and internationally invest in the further digital transformation of justice (Susskind 2017), family justice scholars can take advantage of new opportunities to scale up their study of law in society and, through linking data, capture a more holistic view of child and family journeys through justice and related services.

This article sets out the opportunities and challenges of research using large-scale single, linked and blended administrative datasets for family justice research. By family justice research, we mean in this article to focus on questions about children and families involved with the family courts and related public agencies on account of public or private law matters. Although broader questions of social justice that impact on families can also be addressed using administrative data (for example questions about the distribution of resources), our focus on institutions speaks to growing interest in the UK and further afield on how the wealth of data produced by the family courts can be used to inform the delivery of justice (Brownell and Jutte 2013, Berk et al. 2016, Baldwin et al. 2019, Cusworth et al. 2021).

Many of the points we make in this article will be familiar to researchers in the disciplines of epidemiology and health. However, this article speaks to a community of scholars with more limited experience in the use of large-scale administrative data. Family justice scholars currently produce a wealth of empirical research, but the majority of studies have been based on relatively small samples. That is inevitably (and appropriately) the case for qualitative research that aims to take an in-depth look at processes and experiences, such as the Eekelaar and Maclean series on family justice professionals (Eekelaar et al. 2000, Maclean and Eekelaar 2009, Eekelaar and Maclean 2013). However, it is also the case for quantitative studies in family justice, where there are real challenges to achieving large representative samples, given resource and ethical constraints. Family justice researchers have traditionally made very effective use of court files to advance our understanding across the field. This has included financial remedies (Hitchings et al. 2013, Woodward and Sefton 2014), private law children (Hunt and Macleod 2008, Trinder et al. 2013, 2014, Harding and Newnham 2015) and public law children (Masson 2010). However, to date, this has been based on resource-intensive reading of free text files by researchers, often requiring visits to local courts, rather than the ability to download and analyse anonymised quantitative data. The result is that studies are generally restricted to single snapshots in time, based on a few hundred cases (Masson et al. 2008). The limits of empirical research in family law have been subject to considerable discussion (Brinig 2002, Genn et al. 2006, Broadhurst et al. 2018).

Whilst small n studies and theoretical work are key to our understanding of how family justice works, administrative data is one source of ‘big data’ (Connelly et al. 2016) that can extend the kinds of questions we can ask of our justice institutions. This can include large-scale profiling studies such as who engages with the family courts, and outcome studies about how law shapes lives over time. The paucity of socio-legal research at a population-level persists in a number of international contexts, despite the growth of data safe havens that provide opportunities for the safe use of millions of linked micro-records (Holman et al. 2008, Ford et al. 2009).

Large-scale, data intensive research is an interdisciplinary challenge that requires researchers to work across new knowledge boundaries. Although there is a long history of interdisciplinarity within the family justice research community, when working with large-scale datasets that involve the mass extraction and manipulation of digital data, the interdisciplinary challenge takes on a new guise. Making effective use of large-scale and linked datasets requires multidisciplinary teams, including colleagues with quantitative or computational expertise, which is not typically held by socio-legal scholars. Here, the proliferation of data science institutes is advantageous, facilitating collaborations which enable law and science to more easily align around pressing societal questions. Although novel disciplinary mixes can be as daunting as they are exciting, clarity of roles and equal valuing of expertise makes for productive and more comfortable team work. Family justice scholars can make a distinctive contribution to interdisciplinary data science, by bringing a critical understanding of context to both data collection and analysis. Social theory is far from out of place in this hybrid intellectual space. Robust research using administrative data must be technically robust but also make substantive sense – informed by a deep understanding of the issues that face family justice systems. Learning from colleagues completing theoretical or smaller scale empirical studies, use of big(ger) datasets can address questions of prevalence, equity or trends over time – but only when analyses are infused with a keen understanding of how the law works in practice, in the real world. Empirical family justice research is not simply a technical exercise that seeks to reflect the status quo; rather policy-focused research must be informed by an awareness of the limits of our existing justice institutions and arrangements.

Tailored specifically to the international community of scholars whose work is focused on questions of family involvement and the operation of family justice systems, this article builds on an excellent generic literature on administrative data (Roos et al. 1987, Goerge and Lee 2002, Card et al. 2010, Elias 2014). The first section is largely descriptive, starting with definition, before outlining the value of research using single, linked or blended administrative datasets. The remaining sections of the article speak to questions that our own work indicates are pertinent to family justice researchers. Discussion covers the distinctive contribution of the socio-legal scholar to interdisciplinary data intensive research, before considering the place of the data providers in this collaborative mix. Drawing on the sociological concept of ‘publics’, the final sections of the article consider the multiple interest groups whose social licence must be secured, when secondary analysis uses records produced in the contested world of justice.

Administrative data: definition and value

Definition

Administrative data refers to information about persons or organisational activity, which is collected routinely by government, statutory or other agencies for their own organisational purposes (Woollard et al. 2014). Public institutions, such as the courts and allied services, typically maintain individual case records that not only contain demographic details, but also details of the individual’s interaction with other services. This micro or person-level data is hugely valuable for understanding characteristics of court users, individual pathways through justice systems, legal representation and outcomes of justice involvement. Other data includes information regarding performance, produced to meet the requirements of regulatory bodies or external auditors.

In a number of international contexts, digital administrative records are increasing in volume, quality and longevity, as are opportunities for linking administrative records across sectors of health, education, welfare and justice (Holman et al. 2008, Card et al. 2010). Indeed, a number of proponents have argued that the volume of this data now constitutes a vital source of ‘big data’ (Connelly et al. 2016). Although collected primarily for organisational purposes, this data is increasingly valuable for socio-legal research, given that the shift to digital machine-readable records now delivers millions of person-level records that span a number of years. In England, the HM Courts & Tribunal Service reform programme (The HMCTS reform programme – GOV.UK no date), which seeks to modernise the delivery of justice will deliver a wider a range of new digital resources, resulting from online divorce records. The trend towards digitisation of justice is evident in a number of nation states, as developments in online dispute resolution and adjudication proliferate (Mania 2015).

Capturing family justice populations at scale

So, what are the advantages of this data for justice research? First, administrative data collected or produced centrally by government, the courts and other national agencies, is typically captured at the level of entire service populations, enabling research at a pace and scale which would be very difficult to achieve through other methods. Central to the delivery of family justice is a presumption of even-handed treatment of cases, but in the absence of population-level research, accusations of inequitable treatment of individuals or groups are difficult to either substantiate or refute (Byrom 2019). In this context, retrospective use of institutional records is a practical approach to generating system intelligence and directly addressing professional and public concerns.

To date, family justice researchers have generated a volume of rich qualitative data based largely on interview research with family justice users and professionals. There has also been some use of administrative data in the form of manual searches of court file data. However, the costs of manual collection of data and inputting into digital form for analysis is time-consuming and expensive and inevitably restricts sample sizes.

Whilst qualitative research will always be essential to understand the experiences of individuals and how the family justice system operates, small sample studies can be less persuasive in terms of their influence on policy. In contrast, when researchers combine insights based on entire service populations with qualitative research, even politically unpalatable observations are less easily dismissed (Cusworth et al. 2019). Moreover, whole population data helps researchers overcome difficulties in achieving representative samples. This is particularly important when dealing with contested issues where different interest groups can skew the evidence base, an ever-present issue in family justice research. It is also vital when research aims to engage hard to reach groups (Goerge and Lee 2002, Evans et al. 2010). For example, (Lery et al. 2005) used child protection administrative records to analyse sibling groups within child welfare services, advancing knowledge about the continuation or otherwise of sibling ties, once children enter care. More recently, (Hafekost et al. 2017) examined maternal alcohol use disorder diagnoses and subsequent child protection contact using linked records, identifying health vulnerabilites at scale, in women involved with children’s statutory services. However, examples specific to court populations are fewer in number. (Broadhurst et al. 2015) examined the total volume of records for women appearing in public law care proceedings in England (2007–2014; n = 43,541) to quantify women’s repeat appearances. This latter work catalysed major reform of preventative services following child removal and illustrated, for the first time, the value of adminstrative family court data in England, for population-level research which seeks direct policy impacts. (Harwin et al. 2019) examined the total population of children subject to care proceedings to identify trends in the use of orders (supervision orders, child arrangement orders, special guardianship, care orders, and placement orders) and legal outcomes. This was the first national study to explore child outcomes capturing return to court for further care proceedings for different order types. Among serveral findings, it challenged myths that special guardianship is an unstable permanent placement option for children. These findings have fed directly into policy, with concomitant questioning of practice. Given large sample sizes, administrative data also offers the possibility of creating particular cohorts of individuals to study the impact of policy or legislative change on population subgroups. For example, restrictions in access to legal aid and court reform programmes are apparent in a number of international contexts but with limited analysis of the impact of cutbacks on individual lives or groups (Maclean et al. 2015, Byrom 2019). Powerful analyses published to date might be complemented by the different kind of questions that can be answered through population-level data (Maclean et al. 2015).

Equally, comparative work, which probes the influence of local or regional demographics on family justice, is also possible through disaggregation of total populations. (Harwin et al. 2018) study found marked regional differences in the use of supervision orders supporting reunification and of care orders. Areas that had high rates of supervision orders made proportionately fewer care orders and vice versa, suggesting that cultural practices are important, whilst raising broader issues of equity and fairness (Harwin et al. 2018). The large sample sizes afforded by administrative data also enable quasi-causal analyses, focussing on the different outcomes of family justice interventions on individual lives (Dunning 2012). Innovation in family justice practice is evident in a number of international contexts, but often funding is simply not available to robustly evaluate new practices. In the absence of evaluative data, new and promising practice models often falter, as proponents struggle to evidence the case for sustainable funding. Thus, administrative data is a powerful resource for a range of empirical family justice studies.

Gaining a longer view of law in society

A major advantage of administrative datasets is that they are often longitudinal or can be made longitudinal (Evans et al. 2010). By longitudinal we refer to studies that draw on repeated measures/collection of the same variable, but over an extended timeframe. Conducting longitudinal retrospective analyses based on population-wide administrative data that spans a number of years is a cost-effective method of gaining a long run perspective on organisational and population patterns over time. For example, episodes of individual service engagement are date stamped; they capture an individual’s engagement with services at a particular point in time and any subsequent involvement (see for example, Johnson et al. 2020). Hence, administrative records can be restructured to capture the pathways of individual service users or litigants over time through family justice systems, together with short and longer-term outcomes.

Although epidemiologists and social scientists generally find studies more credible when data collection is prospective and captured according to protocols designed specifically for the research in question, pragmatic considerations must also apply. For example, in the case of the youth courts, it would be very difficult to prospectively track a study population over a lengthy period of time, given the likelihood of significant attrition (Janson 1996). Individuals, youths and families move across geographic borders or may simply drop out of prospective surveys due to changing life circumstances. The likelihood of loss to follow up, which typifies prospective designs, is exacerbated in the case of individuals and families involved with justice systems given the stigmatised nature of problems and personal difficulties (Roos et al. 1987, Brownell and Jutte 2013, Howe et al. 2013). For example, (Harwin and Alrouh 2017) found that parents were reluctant to engage with researchers once children returned home from care, as their focus was on reclaiming as normal a life as possible with their children. Research served as a reminder of a turbulent period of family life that saw parents lose children from their care on account of child protection concerns. However, by extracting and analysing de-identified family court records, (Harwin et al. 2019) were able to answer pressing questions about the strength of the supervision order to support family reunification by calculating breakdown rates. Gaining a longitudinal perspective retrospectively, through the analysis of administrative datasets, can provide a longer view of the outcome of decisions taken in the family or youth courts, without running into such problems.

Joining up the dots: record linkage for justice research

Administrative data produced for non-research purposes is only useful if there are sufficient variables to address research questions (Administrative Data Taskforce 2012). Researchers will typically have little influence over data collection; that is the design, structure and therefore scope of organisational datasets. Core variables that researchers would seek to include in any research study are not necessarily relevant to organisational performance and may therefore not included in organisational datasets. For example, the recording of domestic abuse in cases that concern disputes between parents about children’s residence or time with parents is often poorly recorded, if at all (Cusworth et al. 2020) even though researchers would view these as critically important variables. This is a key limitation of what (Connelly et al. 2016) have described as ‘found’ rather than ‘made’ social science data.

In addition, data quality issues can further restrict the range of variables available for research. Missing data is common in administrative datasets, and although there is a range of principled statistical methods for imputing missing values (Perkins et al. 2018), in some cases the extent of missing values means that information against particular variables cannot reliably be used.

However, the scope of any single dataset can be greatly improved through linking data to other relevant sources, such as health or education datasets. By linking data we gain a less fragmented view of individual interaction with, or pathways through services and the courts (Jay et al. 2017). In addition, there is a wealth of excellent literature on approaches to data linkage (Harron et al. 2017, Zylbersztejn et al. 2020). In the discipline of health in particular, there are multiple examples of robust studies that have linked data across organisational sources to build holistic analyses of patients’ engagement with health services and their outcomes (Harron et al. 2020). For example, (Orr et al. 2019) linked hospital and children’s services records to examine the relationship between hospitalisation for maternal assault and risk of child protection involvement. More recently, Griffiths and colleagues linked Welsh family court records to a number of health records, to uncover very high levels of mental health need in pregnancy for women appearing in care proceedings (Griffiths et al. 2020).

Administrative data can also be linked with longitudinal survey data, such as the Millenium Cohort Study and National Child Development Study. Although historically such data has tended to be used in isolation, there is an increasing volume of published studies illustrating what can be gained by combining these data sources. By linking administrative and survey data we add richness and depth to our analyses. For example, in the US, to examine income as an important measure of wellbeing, (Medalia et al. 2019) combined survey data with tax records, among other sources, to produce the US Comprehensive Income Dataset. In addition, analyses of administrative data can be complemented by prospective data collection, to add depth to understanding. As above, where time and budgets allow, prospective data collection enables the researcher to have greater control over the scope and quality of data collection.

In Scandinavian countries there is a long history of the use of national registers as a major source of administrative information about populations (Connelly et al. 2016). However, in a number of other international contexts, a range of initiatives has now radically changed opportunities for the use of administrative data for secondary analysis. In the UK, successive policy and legislative developments have sought to reduce barriers that stand in the way of linking data across government departments as set out in the ‘UK Strategy for Data Resources for Social and Economic Research’ (UK Data Forum 2013). Administrative Data Research (ADR) UK is an initiative funded by the Economic and Social Research Council which aims to accelerate opportunities for safe linkage of the wealth of administrative data held by governments. Investment in ADR UK is based on the premise that secondary analysis of this data can lead to the efficient production of evidence to inform public policy. In Australia, the Research Data Infrastructure initiative similarly aims to accelerate the use of controlled data (Australian Government Department of Health no date). In Canada, the Canadian Data Platform under the Strategy for Patient-Oriented Research (SPOR) provides a single portal through which researchers can request access to health administrative, demographic and social data (Guttmann 2019).

Indeed, such is the international support for the re-use of valuable administrative data, that funding and research efforts are now being directed towards the sharing of digital micro-data across international boundaries (OECD Expert Group 2014). An increasing recognition that many social issues are global challenges (from climate change to pandemics or human trafficking), means that international co-operation regarding data sharing is ever more pressing. Methods for linking data are also advancing, with a wealth of literature reporting a range of probabilistic techniques to maximise linkage rates and improve linkage quality (Winkler et al. 2015). Such developments are vital, because governments do not typically use common person identifiers across different ministerial departments.

However, it is only very recently that we have witnessed justice-focussed initiatives which aim to accelerate the safe use of linked administrative data. Although there are examples of US studies in particular that have used integrated administrative crime and health records (Parsons and Sandwick 2012), the ability to link person-level criminal, civil or family court records to other data sources has been limited. This may reflect the sensitivity of justice records and that policy responses are highly politicised. However, important new initiatives are emerging in the UK. First, the Ministry of Justice (MoJ) led the creation of an important data-share, joining education and justice data, developing the WATCh Tool (MoJ analysis – Profile | Tableau Public no date) which provides an overview of all children who entered the family justice system between 2010 and 2016 . This was followed by major investment from UKRI ESRC in the MoJ’s Data First programme as part of ADR UK (2020–2023), which has already made available a number of important justice datasets through the Office for National Statistics, with more to follow (Data First: Harnessing the potential of linked administrative data for the justice system – ADR UK no date). The overarching aim of the Data First initiative is to enable independent academic research, to add value and capacity to statistics produced routinely by the MoJ. A key objective of the recently established Nuffield Family Justice Observatory (FJO) is to support the use of linked data for applied family justice research (Broadhurst et al. 2018). Funded by the Nuffield Foundation, the Nuffield FJO is already making its mark in terms of generating vital intelligence about how the family justice system is working in England and Wales. Linked to this initiative, interdisciplinary teams are beginning to produce data resource profiles (Bedston et al. 2020, Johnson et al. 2020) and create data user groups designed to specifically support the family justice research community in England and Wales. Such resource profiles provide practical illustrations of data documentation that can be replicated in other jurisdictions. A number of exemplar studies are evidencing the value of this data to address pressing policy questions (Alrouh et al. 2019, Cusworth et al. 2021).

The relationships of interdisciplinary, data intensive research

The socio-legal contribution

Exploiting large-scale administrative data is an interdisciplinary challenge – it requires subject specialists to join forces with computer and data scientists, statisticians and epidemiologists. Although interdisciplinarity typifies socio-legal scholarship, the particular mix of disciplines required to produce reliable and meaningful research using large-scale, often messy administrative data, is more novel. The linking of data requires knowledge about the architecture of datasets and linkage methodologies. Using this data is far more than a search and count exercise. However, meaningful and accurate use of administrative data also requires specialist knowledge of the specific domain in question. For example, it is difficult for non-specialists to understand the difference between legal orders made in the family courts and their implications for children and families, or to know how to condense the multiplicity of available legal orders for research purposes. Specialist knowledge is also required to identify errors in the data – for example, the combinations of orders recorded for children that are not possible within the law. Equally, subject specialists are needed to identify what administrative datasets cannot answer or what issues may be obscured by gaps in the data (Bryson et al. 2017). The restructuring of any data for research purposes must make both technical and substantive sense.

Working across knowledge boundaries is not always comfortable. Researchers must absorb new vocabularies and norms of working which can leave them less assured of their place or competence. Data intensive interdisciplinary research is not for the faint-hearted, scholars must tolerate a level of not knowing as they adapt their knowledge and understanding (Klein 1990, Lélé and Norgaard 2005). Specialists may also feel that they risk abandoning their core knowledge because keeping abreast of relentless social, political and legislative change is already a huge challenge. However, interdisciplinary research does not typically involve absorbing an entirely new discipline – new or peripheral knowledge complements core knowledge (Palmer 2013). This is a hybrid intellectual space, but one in which the primary discipline will not necessarily suffer as a consequence of integrating new learning.

In this context, it is vital that family justice scholars recognise the distinctive contribution they can make to data intensive research, by sharing their advanced knowledge of the specialist domain. Data do not speak for themselves – questions must be asked of data that are informed by a keen understanding of pressing policy and legal issues. Meaningful interpretation requires in-depth knowledge of a given domain. Where the naïve data scientist simply prospects a new subject domain (Slota et al. 2020), he or she may overlook the impact of key policy and legislative milestones on any time series analysis. As (Slota et al. 2020) write, data science can appear ‘curiously empty’ – where interdisciplinary research is approached as an act of solicitation or simple extraction in a new domain. New developments in population data science aim to avoid such pitfalls. Driven by an over-arching interest in public data for public good, such research groupings bring together individuals with different disciplinary backgrounds to deliver meaningful secondary analyses of administrative data.

Moreover, received concepts in statistics and social science gain new meaning when read through a family justice lens. For example, ‘population reconstruction’ is not simply a practical task of restructuring person-level records. Creating analytic categories is a critical conceptual exercise, which can reinforce or challenge normative structures of society (Fledderjohann and Roberts 2018). For example, social class is on the one hand a way of grouping individuals, but equally is a system of stratification that ranks people by their differential access to material, social and cultural resources. Class shapes lives in important and unequal ways – social theory brings a critical understanding of context to all stages of the design and conduct of research.

From a socio-legal family justice perspective, data is never actually raw – data are collected for specific purposes. Systematic biases in terms of missing data reflect issues in society – note the troublesome category ethnicity. Ethnicity is a loaded category, often poorly completed in administrative records because of sensitivities regarding definition, but also because it is a category that is continually contested and revised. Missing values within datasets can also reflect absence of populations within justice institutions themselves. For example, the recent work of (Bedston et al. 2019), which quantified fathers’ absence in child protection court records in England, uncovered the gendered nature of parents’ recurrent appearances in public law children cases. The frequency with which fathers were absent from family court records could not simply be reduced to problems of recording; rather this absence reflected the marginal role that fathers can play in public law proceedings. The records unintentionally reproduce gender asymmetries. Moreover, no exercise of imputation will remedy this underlying gender bias, embedded in family justice systems.

Large-scale administrative data affords the opportunity of transforming the kinds of questions we can ask of our justice institutions, but use of this data is not and should not be atheoretical (Sarat and Silbey 1988, Monroe et al. 2015, Margetts 2017). The risk of an overly empiricist approach is that it reduces what it means to ‘know’ and to ‘understand’ to levels that are not helpful. Anderson’s (2008) assertion that massive volumes of data ‘forces us to view data mathematically first and establish a context for it later’ is to misunderstand the research process. However, as Boyd and Crawford (2012) caution ‘older forms of intellectual craft’ (p.666) have a central place in data-intensive research, but risk being side-lined where it is simply the sheer volume of data that counts. Working across disciplinary boundaries requires a mindset of mutual respect, which is best fostered through durable teams, rather than fleeting or opportunistic engagements.

Collaborative engagement with data providers

Although administrative data is routinely produced, it is produced by public institutions, which are dynamic in nature. Shifting technical, policy and legislative landscape impact on how data is collected and coded (UK Statistics Authority, 2019). For example, modifications in the classification of offences in criminal justice can cause significant problems where data is simply taken at face value. Boundary changes are also typical in public agencies in regard to their geographic reach. Therefore, effective use of even the most standardised administrative data requires a keen understanding of the operational contexts in which this data is produced (Administrative Data Taskforce 2012). This is a particularly important consideration in the design and piloting of retrospective longitudinal studies that aim to mine data over a number of years, as described above. Thus, researchers must work very closely with those managing organisational data to understand changes in how data is categorised and stored – data owners are essential actors in the collaborative mix.

The interface between disciplines in collaborative teams has been subject to discussion. However, the data provider-academic interface remains far less visible in the published literature. Data providers hold key insights into the unique histories of their institutions’ recording practices, but without opportunities for detailed discussion of methods of data collection and recording, mis-estimation is likely. Although metadata tables produced by institutions enable researchers to unpack an institution’s coding methodology and any changes over time, the devil is in the detail. Close examination of records finds both formal changes to systems but also ‘coding on the fly’. In response to legislative and policy change, agencies need time to make changes to categories of data collection. The immediate management of operations must take priority over concerns about the reliability of archived data for research purposes. Pending changes in data infrastructures, interim strategies or workarounds may be implemented. Without knowledge of informal changes in data recording, the reliable tracking of individuals within systems over time is undermined (UC Data 1999, UK Statistics Authority 2015). Family justice scholars, with specialist expertise of major policy and legislative milestones and awareness of how the law works in practice, are well placed to spot system workarounds where technical infrastructures lag behind formal changes to policy or the letter of the law.

In addition, government departments and related institutions typically produce their own benchmarking data – published both internally and externally, as annual snapshots regarding demand and or performance (Ministry of Justice 2020). Where effective relationships are established with data producers, both sources are important in validating findings. Using an agency’s internal statistics to establish the plausibility of initial descriptive statistics, is an important step in cross-validation (UK Statistics Authority 2015). Checking out observations with data producers also provides an opportunity for augmenting mutual understanding of important administrative data assets and improving their quality and scope. Researchers should not passively accept the limitations of administrative data; rather they should seek out opportunities to improve national data assets.

However, relationships with data owners extend beyond issues of data quality and reliability. With increasing data science capability within government departments and other legal service organisations, research is best conducted in the spirit of collaboration. The academy no longer has a monopoly on data or its analysis. Within organisations, we are witnessing a concerted effort to extract the maximal value from their data. Data cannot be viewed as simply the by-product of routine case management; rather it is a vital source of intelligence for agencies, equipping them to examine their own practices. Thus, an inclusive approach, which explicitly sets out opportunities for mutual exchange and capacity building, will achieve sustainable relationships of co-production. Data acquisition cannot be a simple ‘smash and grab’ exercise. Data producers hold keen intelligence about policy and legislative landscapes upon which they aim to make their mark, and this ensures their central place in any collective research effort.

Administrative data: mixing methods

The use of administrative data for quantitative research does not preclude complementary, qualitative or ethnographic enquiry. Where methods are mixed, considerable impact has followed research, because insights gained from quantitative analysis of large-scale data are enhanced through in-depth interviewing, direct observation of practice or manual review of representative case files. Although examples are limited, specific to family justice research, (Broadhurst et al. 2017, Trinder et al. 2017), in the wider policy literature there are many examples of research which combine methods, adding depth to insights gained from large-scale administrative data, through qualitative approaches. For example, (Grobe et al. 2017) in the US linked administrative records on families and children to data collected through surveys and in-depth interviews to examine employment instability and job characteristics of parents using child care subsidies.

Whilst use of population-level data can identify, for example, stark regional differences in rates of care orders for children, without further qualitative enquiry and deep consultation with stakeholders, explanation of phenomena falls short. Often those using quantitative or computational methods are accused of too much appeal to scientism – however, we would argue that setting different approaches to knowledge generation in opposition is counterproductive. Analyses that make use of large-scale datasets can be complemented by more in-depth qualitative inquiry. Thus, embracing large-scale data intensive research is about the re-assembly of new and ‘old’ methods, rather than the relegation of established repertoires to the history books.

Legal and ethical considerations: towards a social licence for justice research

Increasing access to person-level data raises multiple questions about incursions of privacy. Although use of population-level data can deliver vital societal benefits, these must be carefully balanced with citizens’ privacy concerns. Regarding institutions of justice, if trust is eroded in the security of personal data, this can undermine citizen-state interactions far beyond the life of any particular research programme. Citizen-state interactions in the context of justice, are arguably, fraught with tensions around rights, entitlements, privacy and disclosure, which create particular sensitivities regarding the secondary analysis of personal records.

Considerable progress has been made in the development of robust and transparent models of governance, which control all aspects of data sharing/re-use and storage, data access and analysis, publication and disclosure. The development of secure data sharing platforms, where personal data is anonymised prior to its handling by researchers, demonstrates best practice internationally in privacy protection and information governance (Ford et al. 2009, Lyons et al. 2009, Jones et al. 2019). The socio-legal research community can be assured of best practice, where approved and secure environments support their research.

However, ensuring legal compliance regarding data protection is not the same as securing public support, or what Carter et al. (2015) have referred to as a social licence for the re-use of personal data. An ethical approach to the secondary use of personal data requires more than attention to the legalities of data sharing. Writing on the subject of the care.data debacle in the UK, Carter et al. (2015) argued that where the proponents of this project failed was in securing public trust and public consensus for the secondary use of personal health records. Readers may recall that care.data aimed to extract data from NHS primary care records in England to provide accurate, timely information with the aims of informing all stakeholders about the treatment and care provided by the NHS. The objectives of this service were to improve transparency and accountability within the NHS and, ultimately, outcomes for patients. However, the programme folded due to public objection to the use of personal records. This project illustrates very clearly that having a legal mandate for data sharing, is not in itself sufficient. Legal authority alone does not guarantee social legitimacy. Achieving social legitimacy lies in careful and meaningful engagement with public audiences – over and above simple public relations exercises.

Regarding the community of justice stakeholders, we would add that the sociological concept of ‘publics’ more aptly describes the range of audiences that make up this community (Newman and Clarke 2009). Professionals, advocates, experts, academics and the media are all powerful interest groups with a range of strong opinions. All too often references to public engagement fail to differentiate the diversity of publics who engage with questions of research value and data ethics. Different audiences bring different interests and agendas to their listening and have conflicting interests and values (Lehoux et al. 2012) – nowhere is this more so, than in the adversarial world of justice. While there is clearly a strong moral and economic argument for the safe, anonymous reuse of personal data for research, for all the reasons set out above, the challenge is in engaging with relevant publics, addressing their concerns and demonstrating the value of applied research (Waind 2020). Where public support is not garnered from diverse stakeholder groups, the progress that has been made in harnessing administrative data for research, could readily be reversed. A study conducted by Cameron et al. (2014) commissioned by the ESRC and the Office for National Statistics, in relation to public views on use of administrative data for research purposes, indicated that in general public had very limited awareness of the purpose or value of research, but were concerned about the security of personal data, and worried about data being leaked, lost or sold (Cameron et al. 2014). Therefore, the pursuit of a social licence for the sharing of highly sensitive personal justice data, albeit in safe, fully anonymised formats, must embrace a range of publics – in particular those in receipt of justice services, whose interests are often delegitimised in policy debates.

Caution: pathways to impact

The primary use of large-scale administrative data is to undertake policy and practice focussed analyses. For example, researchers use data generated by our public institutions to study their effects on lives, both in the short and longer term. However, no matter how novel or robust our research derived from administrative data, there is no ‘natural’ pathway between empirical observation and policy or legislative change. Policy makers may be eager for social science solutions to immediate policy issues, but the possibilities for revision are limited – particularly in the context of relentless downward pressure on public service budgets or more recently, economic or epidemiological shocks. Moreover, policy audiences are typically impatient and struggle to comprehend the timeframes for academic delivery, as well as the need to ask more fundamental questions about justice. The pursuit of efficiency gains in the delivery of justice has dominated systems in a number of international contexts, creating practice that is forever in crisis, reactive and, in some instances, exhausted. In search of solutions, the policy lens is therefore downstream on recommendations that may ease this strain, with limited interest in research method. For all these reasons, we cannot assume unproblematic relationships between the producers and receivers of knowledge, or indeed, the parties as set out above.

However, given the academy’s commitment to the public good, there is much value in achieving a compromise between the strong pragmatic impulse of policy and the academic preference for more open-ended critical questions (Campaign for Social Science no date). This requires a balance on all sides between affiliation and distinction, which positions the academic as not in the service of policy, but nevertheless, attuned to the workaday concerns of policy makers and practitioners. Increasingly, research intermediaries seek to address the gap between the worlds of academia, policy and practice. For example, the Nuffield Family Justice Observatory’s (England and Wales) primary focus is to support the production of actionable knowledge and/or to produce accessible summaries for frontline practice.

Discussion and conclusions

Justice systems underpin democracy – they serve to protect but also enable – and play a vital role in combatting abuse and challenging inequality. However, where basic questions about the impact of justice systems remain unanswered, it is difficult to argue that justice systems are doing their job. Using administrative data, particularly where different data sources can be linked, allows the researcher to address critical questions about the impact of justice systems on individual and family lives – and over time.

Although the family justice research community has limited experience in the use of administrative data at scale, scholars can learn from advances within the disciplines of epidemiology and health, in particular. Moreover, family justice scholars can make a fundamental contribution to interdisciplinary data intensive research, because all elements of research design, data collection and analysis require substantive as well as methodological expertise. As stated above, teams that can combine methodological knowledge with theoretically informed social science, are best placed to generate meaningful research about the family justice system.

With the further digitisation of justice through a range of client-facing solutions such as self-help apps, the development of online dispute resolution through to automated adjudication and algorithmic decision-making, it is imperative that family justice scholars keep pace with change. The observations of Savage and Burrows (Savage and Burrows 2007), in their articulation of a crisis in empirical sociology, are all the more prescient. They argued that sociologists risked losing their foothold in the world of social analysis given the proliferation of organisations, which both produce and are equipped to perform sophisticated analyses of their own social data. Moreover, are family justice scholars happy to leave analysis of ‘big data’ to those who contend like Anderson (Anderson, 2008) that theory is dead? Or do they wish to ensure that explanation is as important as actions of quantification and the like? Forming strong alliances with data producers as well as colleagues across a range of disciplines will enable the family justice community to make its mark, as research opportunities continue to transform in the context of an expanding landscape of digital justice data.

Funder

Nuffield Foundation: grant number FJO/43,766

No involvement in any stage of the article production – only funding

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Nuffield Foundation [FJO/43766].