Knowing from where to start: a plea for more and better systematic literature reviews about the effectiveness of legal rights for crime victims

ABSTRACT

Victimologists have often called for more empirical research into the effectiveness of legal rights for crime victims. In this article I contend that it is currently too early to heed such calls due to the lack of insight into prior research addressing this topic. I argue that systematic literature reviews can help us to get this insight, particularly if they adhere to the principles of the realist synthesis approach. A core strength of this approach is that it enables the interpretation of results from statistical analyses in relation to the theoretical mechanisms underlying the legal rights of interest and the contexts in which they have been implemented. This is not only important to set a sound agenda for future research, but also to direct the development and implementation of these rights and, eventually, to have crime victims truly benefit from them.

KEYWORDS:

• Crime victims
• legal rights
• effectiveness
• systematic literature reviews
• realist synthesis approach

1. Introduction

To deal with the consequences of crime, criminal justice systems have traditionally focused on the punishment of offenders. However, since the 1960s many Western criminal justice systems have become more victim oriented. This shift in focus from the offender to the victim is largely due to increases in crime rates and the emergence of victim activist groups which have successfully lobbied for legislative reforms (Young & Stein, 2004; Waller, 2010; Groenhuijsen, 2014). Incited by these social developments and international and supranational obligations, such as imposed by the United Nations Declaration of Basic Principles of Justice for Victims of Crime and Abuse of Power of 1985, the EU Council Framework Decision on the standing of Victims in Criminal Proceedings of 2001, and the EU Directive establishing minimum standards on the rights, support and protection of victims of crime of 2012, many countries around the Western world have passed laws that entitle crime victims to such rights as receipt of adequate information, compensation, support services (practical, emotional, medical, and legal), participation in criminal proceedings, correct treatment by legal professionals, and protection against offenders (see Brienen & Hoegen, 2000; Van der Aa et al., 2009; Wilson & Ross, 2015; Tobolowsky, Beloof, Gaboury, Jackson, & Blackburn, 2016; Kirchengast, 2017; Perrin, 2017). Although these rights typically aim for different specific outcomes, they also seem to share the following general goals: (1) acknowledgement (e.g. of crime victims’ suffering and their legitimate role in the criminal justice system), (2) restoration of material and intangible damages, (3) prevention of secondary victimisation (i.e. the aggravation of suffering by crime victims’ involvement in legal procedures; cf. Montada, 1994), and (4) prevention of revictimisation (Groenhuijsen, 2008, 2014).

Many victimologists have criticised the recent legal reforms, arguing that it is unknown whether the legal rights that have been developed to meet the needs of crime victims are effective in reaching their goals, because very little empirical research has addressed this topic (e.g. Fattah, 1999; Sebba, 2001; Lehner-Zimmerer, 2011; Wemmers, 2013; Biffi et al., 2016). According to Biffi et al., “the lack of a sufficient evidence base” entails “a serious impediment for our ability to provide victims with effective care, support and protection and rights that contribute to their well-being and interests” (Biffi et al., 2016, p. 200). They therefore argue that “a full-fledged effort to secure this evidence base (by conducting more empirical research), in particular in those areas where it is non-existent, should be taken as a priority” (Biffi et al., 2016, p. 200).

Contrary to what Biffi et al. (2016) have argued, I contend in this article that it is currently too early to heed calls for more empirical research, since we lack insight into empirical research evaluating the effectiveness of legal rights for crime victims (see also Lauritsen & Archakova, 2008; Lauritsen, 2010; Fox & Shjarback, 2016). We therefore do not yet know what is already known and what requires (further) investigation (cf. Petticrew & Roberts, 2006). Filling this knowledge gap is not only of utmost importance to set a sound agenda for future research and to know from where to start, but also to direct the development and practical implementation of legal rights for crime victims and, eventually, to ensure that crime victims will truly benefit from these rights. In the next paragraphs I will explain how this can be achieved.

2. Resolving the knowledge gap

To resolve the current knowledge gap, we first need to extend the number of legal rights for crime victims which have been subjected to a systematic literature review (Lauritsen, 2010; Fox & Shjarback, 2016). In a systematic literature review multiple literature databases covering a broad range of scientific disciplines are searched in a highly structured manner. The goal of this search process is to identify empirical studies which are relevant to answering a particular research question. To make sense of the large body of information provided by selected studies and, eventually, to answer the research question of interest, identified studies are usually appraised and synthesised according to a predetermined protocol (Petticrew & Roberts, 2006).

Systematic literature reviews are rare in the field of Victimology. It is therefore difficult to obtain and maintain a complete picture of victimological research and to base new initiatives on scientific knowledge (Lauritsen, 2006, Lauritsen & Archakova, 2008). This also applies to that part of Victimology which is devoted to legal rights for crime victims; to date, very few legal rights for crime victims have been subjected to a systematic literature review about their effectiveness. An exception applies to protection orders imposed on offenders of domestic violence in order to protect their victims against revictimisation. Several systematic literature reviews have evaluated the effectiveness of such orders and have found that offenders frequently violate them (Logan, Shannon, Walker, & Faragher, 2006; Benitez, McNiel, & Binder, 2010; Russell, 2012; Dowling, Morgan, Hulme, Manning, & Wong, 2018). Subjecting other legal rights for crime victims to a systematic literature review is necessary to gain insight into their effectiveness as well. However, conducting more systematic literature reviews will not be sufficient to fill the current knowledge gap; we also need to overcome the limitations of the few available systematic literature reviews. This requires four efforts. First, we need to make sure that future systematic literature reviews search for studies which assessed effectiveness in terms of intended goals. Prior systematic literature reviews have often searched for studies that used victims´ satisfaction with the criminal justice system as an indicator of effectiveness (e.g. Laxminarayan, Bosmans, Porter, & Sosa, 2013; Kunst, Popelier, & Varekamp, 2015; Wedlock & Tapley, 2016), but satisfaction, due to its generic meaning, is not an appropriate outcome measure to assess legal rights´ effectiveness (cf. Biffi et al., 2016). For example, whether victims feel satisfied with their participation in criminal proceedings does not necessarily indicate that acknowledgement – an important general goal of crime victims´ right to participate in criminal proceedings – has been achieved as well. Second, we need to make sure that future systematic literature reviews also search for studies that investigated unintended outcomes. For example, when reviewing research on outcomes of compensation, we should not only look into studies which assess such intended outcomes as restoration of crime victims’ financial status, but as well into those which assessed such unintended outcomes as deliberate symptom over-reporting (to increase the chance of receiving compensation for intangible damages). This is an important outcome to take into consideration, since compensation should only be granted for damages which have truly occurred (see also Kunst, Winkel, & Bogaerts, 2011; Kunst & Winkel, 2015). Third, we need to make sure that future systematic literature reviews extend their search processes to databases that include non-English and grey literature. To date, most systematic literature reviews have confined their search processes to publications written in English and articles published in scholarly journals. Such restrictions are extremely problematic for getting a comprehensive and thorough insight into available empirical knowledge, because relevant studies may have been published in other languages than English and are not always included in scholarly journals, especially when they did not obtain statistically significant results (Chan & Altman, 2005; Song et al., 2010). Neglecting these studies may result in a severe misrepresentation of available research (Reed & Baxter, 2009; Mahood, Van Eerd, & Irvin, 2014). For example, Wedlock and Tapley (2016) have recently reviewed the literature on the effectiveness of victim support services, but overlooked several relevant Dutch publications (e.g. Zeilstra & Van Andel, 1990; Steinmetz, 1990) because of their focus on studies written in English. As a result, they, in my view, incorrectly concluded that timely and accurate information “assist(-s) victims in coping with the impact of victimisation” (Wedlock & Tapley, p. 5). If they had taken into account the Dutch studies – which indicated that information provision does not help victims in their emotional recovery, they would probably have come to another conclusion. To avoid such misinterpretations of the literature in the future, I particularly recommend future systematic literature reviews to search for studies from such countries as Germany, Belgium, France, the Netherlands, Spain, and the Nordic countries; these countries have rich traditions in both victim legislation and victimological research (see Brienen & Hoegen, 2000; Van der Aa et al., 2009; Sigfridsson, 2011; Dussich, 2015; Daigle, 2018). Fourth, and contrary to most existing systematic literature reviews, we need to make sure that future systematic literature reviews infer effectiveness not from effect evaluations alone, because effect evaluations typically rely on a successionist approach to establish causality (and, therefore, effectiveness). This approach is based on a positivist view of science and argues that knowledge can only be obtained by empirical research of the observable world and assumes that causality can only be established by experimentally manipulating variable X and observing an effect on variable Y (Pawson, Greenhalgh, Harvey, & Walshe, 2005). Unfortunately, effect evaluations which investigate the effectiveness of legal rights for crime victims usually need to rely on quasi-experimental designs, because it is either legally prohibited or ethically incorrect to randomise some participants to a condition in which they cannot use their legal rights. Moreover, even if random allocation of participants is possible, it will often be very difficult to rule out potential confounding by external factors. For example, the quality with which legal rights have been implemented in criminal procedures may influence the extent to which intended or unintended outcomes are achieved. Information about implementation quality is, however, usually very difficult to quantify and therefore unsuitable for inclusion in statistical analyses. Given these caveats, I think future systematic literature reviews about the effectiveness of legal rights for crime victims need to do two things: (1) they need to use a more substantive approach to synthesise findings from prior research and draw conclusions about effectiveness and (2) they need to include more types of evaluation studies in their review process to succeed in such an approach. The first need can be met by using the realist synthesis approach and the second by the inclusion of programme and process evaluations in the review process.

2.1 The realist synthesis approach in a nutshell

The realist synthesis approach has been developed by Pawson et al. (2005) and is based upon Pawson and Tilley (1997) seminal work about realist evaluation. Contrary to researchers who hold a positivist view of science, realist researchers adhere to a critical realist view of science and argue that knowledge can also be obtained by forming theories about the unobservable world (Pawson, 2013). In particular, realist researchers argue that an exclusive focus on a study´s internal validity (i.e. whether and to what extent the study’s methodological design allows for drawing conclusions about cause and effect) – as is typically done in traditional evaluation research – is not very informative when evaluating the effectiveness of what Pawson and Tilley (1997) call “social interventions”, that is, interventions which are implemented in real-life instead of artificial laboratory settings, such as public policies and programmes, but also laws and the rights they provide to particular groups or individuals (see Klein Haarhuis & Niemeijer, 2009; Leeuw & Schmeets, 2016). This is due to the nature of such interventions. According to Pawson and Tilley (1997), social interventions are characterised by four features: they are (1) theories, (2) embedded, (3) active, and (4) parts of open systems (Pawson & Tilley, 2004). In my view all these features also apply to many of the legal rights adjudicated to crime victims. Let me explain this in more detail on the basis of the right to deliver a victim impact statement. The feature “theories” means that social interventions are rooted in ideas about the causes of inappropriate or unwelcome behaviour, discriminatory events, or inequalities in social conditions and speculations on how these problems can be solved. For example, in the Netherlands, the main reason to introduce victim impact statements was to facilitate crime victims’ emotional recovery; the Dutch legislator assumed that many crime victims suffer from emotional problems and would benefit from speaking during court proceedings (Lens, Pemberton, & Groenhuijsen, 2010). Only if this assumption is valid, does it seem reasonable to expect a therapeutic effect from victim impact statement delivery. The feature “embedded” means that social interventions are implemented in a particular context. The success or failure of a social intervention therefore heavily depends on the characteristics of this context. For example, whether a victim impact statement will facilitate crime victims’ emotional recovery may depend on the presence of social support figures during the delivery of the statement. The feature “active” is related to the second and means that the effects of social interventions are produced by and require the active engagement of actors involved in the implementation process and members of the target population. For example, whether victim impact statements help victims to recover emotionally from the crime depends on their willingness to speak about the crime during the court hearings and the courtesy of the defence (see Lens et al., 2010). The final feature – “parts of open systems” – is also related to the second and means that social interventions are constantly subject to revision and improvement. For example, if victim support workers think that forms to submit requests for delivering victim impact statements are difficult to complete, such forms are likely to be simplified to ease the application procedure.

Given the characteristics of social interventions, the purpose of a realist synthesis is not so much to infer causality (and effectiveness) between an intervention and the outcome(-s) of interest on the basis of prior studies´ methodological designs, but to “discern(…) what works for whom, in what circumstances, in what respects and how” (Pawson et al., 2005, p. 21). This is achieved by studying combinations and interactions of contexts, underlying mechanisms, and outcomes. Studying such context-mechanism-outcome (CMO) configurations is necessary to adequately interpret statistical results from prior research and understand why a social intervention works well at one moment or in one target group or locality, but not (well) at other moments or in other target groups or localities (Pawson, 2002a).

Without a sound knowledge of CMO configurations, it is very difficult, if not impossible, to interpret results from included studies, particularly when these studies provided mixed or inconsistent results. Let me explain this on the basis of a recent systematic literature review by myself and my colleagues on the therapeutic outcomes of crime victims’ satisfaction with their participation in criminal proceedings. This review produced inconsistent results (Kunst et al., 2015). We did not know how to interpret this finding. I now think that an important reason for our ignorance is that we did not look for the information necessary to (re-)construct the CMO configurations of the modes of victim participation evaluated by the studies included in our review. If we would have looked for this information, we might have been able to identify circumstances under which or groups of crime victims for whom satisfaction with participation in criminal proceedings has positive, negative, or null effects (cf. Klein Haarhuis & Niemeijer, 2009; Leeuw & Schmeets, 2016). For example, we might have been able to show that satisfaction with participation in criminal proceedings results in therapeutic outcomes for crime victims who feel empowered by their participation, in anti-therapeutic outcomes for crime victims who feel disempowered by their participation, and neutral outcomes for crime victims who feel neither empowered nor disempowered by their participation (cf. Garvin, 2012).

2.2 Application of the realist synthesis approach in previous systematic literature reviews

The realist synthesis approach has frequently been used in previous systematic literature reviews, particularly in the domains of health care, education, management, and public safety (Berg & Nanavati, 2016). A major limitation of its application in previous systematic literature reviews is that many of them have failed to adhere to the core principles of the realist synthesis approach. This also accounts for the few systematic literature reviews that used this approach to study the effectiveness of social interventions developed for crime victims. For example, inspired by Van der Knaap et al. (2006, 2008), I myself and my colleagues have combined the realist synthesis approach with the Campbell Collaboration Crime and Justice standards to systematically review the effectiveness of revictimisation prevention measures (Kunst, Van Dijk, Pemberton, & Bruinsma, 2008). In this study we restricted the inclusion of prior research to effect evaluations with a score of 3 or higher on the Maryland Scientific Methods Scale (Sherman et al., 1997) and based our assessment of CMO configurations on the introduction and discussion sections of the respective publications. This means that our synthesis of the literature was confined to methodologically rigorous quantitative effect evaluations. This is problematic, since such studies typically provide very little information about the underlying theoretical mechanisms of interventions and the contexts in which they have been implemented. It was therefore only to a limited extent able to do what the realist review approach is all about: to provide insight into what works for whom, in what circumstances, in what respects, and how (cf. Pawson 2013).

2.3 Applying the realist synthesis approach in future systematic literature reviews

Systematic literature reviews using the realist synthesis approach should consider all kinds of studies that might be relevant for the (re-)construction of CMO configurations. For that reason it is not appropriate to confine the search process to studies which have quantitatively assessed the outcomes of a particular intervention and fulfill particular methodological criteria (Pawson & Tilley, 2004; Pawson et al., 2005; Pawson, 2006), as is usually done in systematic literature reviews claiming to have used this approach. Nevertheless, some kind of demarcation of the literature search process is necessary (Pawson et al., 2005). To allow for discerning what works for which victims, in what circumstances, in what respects, and how, I recommend future systematic literature reviews using the realist synthesis approach to include the following three types of evaluation studies:

• Programme evaluations. This type of evaluation study is typically conducted to estimate the potential effectiveness of an intervention on the basis of the plans at hand (Leeuw, 2003; Van Ooyen-Houben & Leeuw, 2010). Inclusion of programme evaluations in systematic literature reviews is needed to specify the goals which have been set for the legal rights under investigation, to unravel the mechanisms assumed necessary to achieve these goals, and to identify the empirical and theoretical literature on which these assumptions were based. A recent study by Muraya and Fry (2016) provides an example of how this can be accomplished. They reviewed fifteen programme evaluations about aftercare services for child victims of trafficking – which are a specific type of victim support and aim to adequately treat the negative psychological, social, and physical outcomes of child trafficking. The results of this review suggested that, in order to benefit fully from such services, “aftercare services need to have comprehensive case management systems as well as multidisciplinary, multiagency, and, where necessary, multinational coordination of efforts” (Muraya & Fry, p. 204), because this ensures the “continuity (of) the emotional support for the child” (Muraya & Fry, p. 211). In other words, continuity of emotional support is deemed an important mechanism in addressing the negative outcomes of child trafficking.

• Process evaluations. This type of evaluation study is typically performed to determine whether an intervention is being or has been implemented according to its plans (by investigating what was done, how, and by whom). Inclusion of process evaluations in systematic literature reviews is necessary to identify context features that may interfere with an adequate implementation of legal rights for crime victims (Wartna, 2005; Nas, Van Ooyen-Houben, & Wieman, 2011; Van Ooyen-Houben, Nas, & Mulder, 2011). Greeson and Campbell (2013) provide an example of how this can be accomplished. They reviewed six process evaluations of sexual assault response teams (SARTs) – also a specific type of victim support – in the United States. An important result of this review was that at several locations confidentiality of information had restricted information sharing between team members. This is problematic, because SARTs were originally developed “to increase the collaboration and build positive relationships among the systems that respond to sexual assault, particularly the legal, medical, and mental health/advocacy systems” (Greeson & Campbell, 2013, p. 84). In other words, contexts which are characterised by a lack of information sharing pose a threat to the implementation of SARTs.

• Effect evaluations. This type of evaluation study is typically carried out to assess the success or failure of interventions. Inclusion of effect evaluations in systematic literature reviews is necessary to verify whether legal rights for crime victims have been successful in reaching their intended outcomes (Lipsey, 2009: Van Ooyen-Houben & Leeuw, 2010). My own work provides an example of how this can be accomplished. In the 2008 study, to which I referred in the previous paragraph, my colleagues and I reviewed, among other things, seven effect evaluations of single information sessions – which are a specific type of information provision – provided to female victims of sexual assault with the aim to prevent revictimisation (see Kunst et al., 2008). The results of this review suggested that such information sessions are not effective in the prevention of revictimisation. In other words, single information sessions do not reach their intended outcome.

An example

How can the realist synthesis approach enrich our understanding of the effectiveness of legal rights for crime victims? Suppose we want to assess the effectiveness of victim impact statements. Victim impact statements allow victims and bereaved family members to tell criminal justice decision makers how the crime has affected their lives, usually with the intention to help them emotionally recover from the act of violence (Erez, 1994). If we would conduct a systematic literature review of studies published in scholarly journals to identify research that examined the achievement of this outcome, only one study would be worth discussing: an effect evaluation by Lens et al. (2015). On the basis of this study, we would conclude that victim impact statement delivery does not result in emotional recovery. Indeed, it was found that victim impact statement delivery was associated with a slight increase in anger and anxiety symptoms, which suggests that it had a retraumatisation effect. According to the authors, their findings are in line with “ongoing research into the social sharing of emotions, which reveals that the mere expression of emotions has no direct ‘healing’ effects (Rimé, 2009; Rimé, Kanyangara, Yzerbyt, & Paez, 2011), and research on post-traumatic stress that challenges the assumption that a single-shot expression of emotions contributes to a diminishing of trauma complaints (Van Emmerik, Kamphuis, Hulsbosch, & Emmelkamp, 2002)” (Lens et al., 2015, p. 14). Although these contentions may well be true, they do not explain why a retraumatisation rather than a therapeutic effect was found.

We would yet be able to provide a sound explanation for the findings of the Lens et al. (2015) study by extending our systematic literature review to non-English publications and grey literature. That extension would yield two additional but Dutch publications: a research report by Lens et al. (2010) and a journal article by myself (Kunst, 2015). The first publication – to which we also referred in paragraph 2.1 – formed the basis for the Lens et al. (2015) study and provides insight into the specific context of victim impact statement delivery in the Netherlands and the obstacles crime victims meet when they want to deliver such a statement. In other words, it also includes a process evaluation. The second publication provides a critical commentary on the assumed mechanism through which victim impact statements in Dutch criminal proceedings should, according to the Dutch legislator, result in emotional recovery. It can therefore be seen as a programme evaluation. On the basis of these additional studies, we would still come to the conclusion that victim impact statement delivery does not result in emotional recovery, but we would also be able to explain why this is the case, as the two studies would provide us with the following additional information:

• The programme evaluation by myself (Kunst, 2015) would teach us that victim impact statement delivery needs to take place within a few weeks after the crime to produce a potentially therapeutic or – more correctly stated – preventive effect. Most crime victims return to their pre-crime level of emotional wellbeing during this period of time and therefore will not benefit from speaking during the court hearing (cf. Kunst & Koster, 2017). The small proportion of crime victims that does not return to its pre-crime level of emotional wellbeing will, on the other hand, only benefit from clinical interventions. The programme evaluation would additionally teach us that it is not so much victim impact statement delivery per se that promotes victims´ emotional recovery from the crime, but rather their satisfaction with how they were treated by other participants of the criminal proceedings during the court hearing (cf. Kunst et al., 2015; see also Kunst, Rutten, & Knijf, 2013).

• The process evaluation by Lens et al. (2010) would teach us that victim impact statement delivery in Dutch courts usually occurs after substantial lapse of time; on average, study participants who had delivered a victim impact statement did so 17 months post victimisation. In addition, it would teach us that crime victims are sometimes forgotten by the chairing judge and therefore deprived from their right to speak.

On the basis of this additional information, it seems reasonable to argue that the authors’ failure to document a therapeutic effect is particularly due to the meagre empirical underpinning of its assumed working mechanisms and its unsatisfying implementation. An obvious recommendation in view of the programme and process evaluation would be to bring the delivery of victim impact statement delivery forward to an earlier time during the criminal justice process and to ensure its adequate implementation. Only under these circumstances does it seem reasonable to expect a therapeutic effect from delivery of a victim impact statement for some victims.

3. Conclusion

The field of Victimology is scattered across many different scientific disciplines (Lauritsen, 2006). It is therefore difficult to obtain and maintain a complete picture of victimological research and to base new initiatives on scientific knowledge (Lauritsen & Archakova, 2008). To overcome these problems, several scholars have pleaded for systematic literature reviews that take into account the methodological and theoretical strengths and weaknesses of available research (e.g. Lauritsen, 2010; Fox & Shjarback, 2016). However, to date, these pleas have largely been ignored. This also accounts for the study of the effectiveness of legal rights for crime victims. In this article I contended that it is therefore too early to heed calls for more empirical research. Before we do that, we first need to gain a comprehensive and more thorough insight into the current state of the literature. I argued that this can be achieved by conducting more systematic literature reviews and learn from the limitations of existing systematic literature reviews. In particular, we need to ensure that (1) effectiveness is operationalised in terms of intended outcomes rather than in terms of satisfaction, (2) unintended outcomes are considered as well, (3) non-English publications and grey literature are searched for relevant studies, and (4) use an approach which does not assume that effectiveness can only become apparent in experimental research. Instead a more substantive approach is needed to determine effectiveness. An approach that meets this need is the realist synthesis approach. In a realist synthesis results from statistical analyses are interpreted in relation to the theoretical mechanisms underlying the intervention of interest (in our case a legal right for crime victims) and the contexts in which this intervention has been implemented. To obtain sufficient information about these underlying mechanisms and contexts, I recommended not to rely on effect evaluations alone and also make use of programme and process evaluations. Programme and process evaluations typically contain much more information about context(-s) and its underlying mechanism(-s) than an effect evaluations and therefore better allow for discerning what works for which victims, in what circumstances, in what respects, and how than effect evaluations alone (cf. Wartna, 2005). This is not only important to set a sound agenda for future research and to know from where to start, but also to direct the development and practical implementation of such interventions and, ultimately, to ensure that crime victims will truly benefit from the legal rights that have been developed to their benefit (cf. Pawson, 2002a, 2002b; Pawson et al., 2005).

Additional information

Notes on contributors

M.J.J. Kunst

M.J.J. Kunst, PhD, LLM, is a Full Professor of Criminology at Leiden University, The Netherlands. His research focuses on the psychological impact of violent crime victimisation, the effectiveness of legal rights for crime victims, and judicial decision making about victim credibility. He teaches an introductory course on Victimology in the bachelor Criminology and the course Forensic Victimology in the master Forensic Criminology.