Legal psychologists as experts: guidelines for minimizing bias

ABSTRACT

Legal psychologists’ assessments can have a major impact on the fact finder’s evaluation of evidence and, consequently, perceptions of guilt. Yet, in the few studies about legal psychologists’ assessments and reports, great variability was found. As is the case with other forensic expert domains, legal psychologists are prone to cognitive biases, such as being adversely affected by irrelevant contextual information, confirmation bias, and allegiance bias. Based on the scientific literature, we propose several ways in which legal psychologists can minimize cognitive biases in their assessments, most notably the alternative scenario method. Furthermore, we propose guidelines for expert witnesses in the legal psychological domain, designed to make reports as scientifically grounded, applicable, readable, transparent, and bias-free as possible. We hope that the guidelines will enhance the quality of expert witness testimony provided by legal psychologists around the world.

KEYWORDS:

• Court expert
• expert witness
• legal psychology
• cognitive bias
• confirmation bias
• forensic decision-making

In the vast majority of criminal cases, testimonies from victims, eyewitnesses, and suspects constitute a primary source of evidence (e.g. Davis et al., 2014; Kebbell & Milne, 1998). The interpretation of such testimonial evidence is not always straightforward. Questions often arise about issues such as the reliability and validity¹ of eyewitness testimony, the manner in which line-up identifications have been conducted, or the value of a confession. All of these questions fall within the domain of legal psychology, a branch of psychology that concerns psychological processes in legal contexts – ranging from cognitive to social processes.

Lawyers, prosecutors, or judges may ask a legal psychologist to comment on a variety of issues. The evaluation and testimony from these experts can have a major impact on police investigations, as well as on a judge’s or jury’s decision to convict or acquit (see e.g. Blandon-Gitlin et al., 2011; Griffith et al., 1998; Loftus, 1980). However, relatively little is known about the quality of legal psychologists’ expert testimony. In the present article we review the relevant literature on expert evaluations, highlight cognitive biases to which experts may fall prey, and propose a set of guidelines for legal psychologists, with the goal of promoting well-founded, transparent, and applicable expert assessments in which evidence-based measures are taken to limit the pervasive influence of cognitive biases.

The quality of psychological expert assessments

Because expert assessments in the legal psychological domain can have such a notable impact on legal decisions, it is important to assess their quality, that is, the extent to which the assessments adhere to scientific standards, represent the state-of-the-art research findings, are applicable to the case at hand, and minimize cognitive bias. Although there is an extensive literature on forensic expert work more generally (for reviews, see Cooper & Meterko, 2019; Kukucka & Dror, 2022), studies directly assessing or discussing assessments made by legal psychological experts are limited. In fact, we are only aware of seven (case) studies on this issue in the legal psychological domain (Brackmann et al., 2016; Gumpert & Lindblad, 2000, 2001; Gumpert et al., 2002a; Nierop et al., 2006; Otgaar et al., 2017; Zajac et al., 2013). Before we delve into each of these studies, we discuss research from a related domain that may provide relevant insights, namely, forensic psychology.

Insights from forensic psychology

Although forensic psychology is sometimes defined, for example in the United States, as a broad domain that includes areas of expertise that we would consider legal psychology,² we use the term to denote a narrower domain as it is commonly known in many European countries. Specifically, forensic psychology is distinct from legal psychology in that it usually concerns a clinical assessment of psychopathology, risk, harm, or competence of individuals, rather than an assessment of cognitive and social factors potentially affecting statements and decisions (cf. Netherlands Register of Court Experts, 2020). Consequently, the major difference lies in the type of work that is done. Where the forensic psychologist can have intensive contact with a suspect, eyewitness, or victim, the legal psychologist mainly uses the case file and recordings of, for instance, investigative interviews. However, the two domains also share some common ground; for example, they both involve evaluation of statements.

The quality of forensic psychological testimony has been examined in several countries. For example, in Ireland’s (2012) study, 126 expert psychological reports from family court proceedings in the United Kingdom were evaluated by four independent registered clinical or forensic psychologists. Two thirds of the reports were rated as poor or very poor on overall quality (p. 24). Likewise, Da Silva Guerreiro et al. (2014) concluded that most of the 106 forensic psychological reports in their Portuguese sample failed to meet the basic criteria for relevance (e.g. use of a clear methodology and various sources of information) and coherence (e.g. information being presented in a logical manner, explanations following from previous information). Nicholson and Norwood (2000) compared six studies on the quality of criminal forensic reports in the United States and found that quality fell ‘far short of professional aspirations for the field’ (p. 9), for example, for the use of psychological testing and third-party information.

In Australia, Doyle et al. (2011) found that although valid risk assessment methods were commonly used and experts tended to agree on the risk assessment outcome, there were also cases in which invalid risk assessment methods were used or valid methods were misapplied and misinterpreted, concluding that ‘the standard of practice of risk assessment must be raised’ (p. 547). In an exploratory study, Bycroft et al. (2019) found that despite the high degree of consensus among Australian experts involved in forensic evaluation of juvenile offenders, experts disagreed on the use of risk assessment tools and none were able to describe their decision-making process. Schimmel and Van Koppen (2017) examined a sample of 1,074 psychological tests administered by forensic psychologists in The Netherlands and found that only 47% of the tests were deemed of sufficient quality by the official Dutch test evaluation committee. Similarly, Neal et al. (2019) found in the United States that only 40% (n = 91) of the studied psychological tests used by forensic psychologists in courts received favorable reviews in terms of their psychometric properties.

In sum, empirical studies on the quality of (clinical) forensic psychological reports across different countries come to remarkably similar conclusions: There is plenty of room for improvement. With regards to the quality of legal psychological expert assessments, we could find only seven (case) studies, conducted in three countries: Sweden (Gumpert & Lindblad, 2000, 2001; Gumpert et al., 2002a), The Netherlands (Brackmann et al., 2016; Nierop et al., 2006; Otgaar et al., 2017), and New Zealand (Zajac et al., 2013). We consider each of these in turn below.

Sweden

Gumpert and Lindblad (2000) performed a qualitative analysis of ten expert witness reports that used the Swedish version of statement analysis. They observed several differences in the way the experts judged certain case characteristics. For example, one expert deemed the alleged victim’s lack of affect a sign that the statement was not valid, whereas another expert thought the exact opposite. The authors noted that it was unclear if these differences were due to ‘appropriate adjustment to the individual cases’ or ‘a lack of consensus among experts’ (p. 301). However, in some cases, the presence of complicating contextual information seemed to change the way experts interpreted the statement, risking a biased conclusion.

In another qualitative study, Gumpert and Lindblad (2001) analyzed Swedish court files of child sexual abuse cases in which an expert witness evaluated the credibility or reliability of the child or their statements. They found that the experts differed greatly in their methodology and descriptions thereof. For example, some experts based their evaluation on the existing case file, whereas others collected their own data. Some experts described the statements of the child in great detail, whereas others exclusively mentioned that the child had disclosed abuse. References to scientific literature were only sometimes included. Furthermore, the authors noted that experts tended to interpret the word ‘credibility’ in different ways, where sometimes it referred to the child’s character and other times to the quality of a statement. They concluded that these differences in procedure had attributed to miscommunications between courts and experts.

In a quantitative study, Gumpert et al. (2002a) investigated the quality of written expert testimony in child sexual abuse cases in Sweden. They developed an assessment tool called the Structured Quality assessment of eXpert testimony (SQX-12; Gumpert et al., 2002b), which consists of twelve criteria evaluating expert witness reports in terms of ‘formal aspects’ (e.g. competence of the expert, sources of information) and ‘statement content’ (e.g. description of the allegations, alternative explanations). They rated 121 expert witness reports on a 3-point scale for each criterion ranging from 0 (absence of criterion) to 2 (presence of criterion) with 1 indicating that the criterion was partially present. The authors found that expert testimony generally did not meet the recommended guidelines. More specifically, 50% of reports received a score indicating clear low quality (i.e. scoring 13 or less out of 24 points), while only 16% of reports received a sufficient score (i.e. scoring 19 or more out of 24 points).

The Netherlands

Nierop et al. (2006) conducted a qualitative analysis of expert witness reports written by six legal psychologists about testimonies in sexual abuse cases in The Netherlands. In these reports, the experts used either Statement Validity Assessment (i.e. a tool for determining the validity of a statement) or the alternative scenario method. The authors concluded that the way in which experts applied the same method varied greatly. For example, one expert discussed all of the Validity Checklist criteria, whereas another discussed only six. Another expert reformulated the existing criteria and replaced some with their own. Experts who used the alternative scenario method described varying numbers of scenarios, ranging from one to seven. One expert did not describe any alternative scenarios but merely mentioned the possibility of alternative scenarios. In addition, the authors concluded that the experts’ conclusions and their form varied widely. Of those who used Statement Validity Assessment, one expert used a 5-point scale to judge the credibility of the statements, whereas another expert provided only an overall credibility judgment. Another expert drew up a list of pros and cons and left the credibility judgment up to the judge. Of those who used the alternative scenario method, some experts contradicted themselves by judging the credibility of the statements while stating that weighing the evidence is the responsibility of the judge.

Brackmann et al. (2016) described a Dutch case of a young child claiming to have witnessed the murder of her mother. In this case, two expert witness reports were written by psychologists, which differed dramatically in content and conclusion. The first expert, hired by the public prosecutor, concluded that there was no strong evidence to question the validity of the witness statement. The second expert, hired by the defense, concluded that the witness memory was a false memory, and therefore the testimony was not valid. Interestingly, both experts used the same method of alternative scenarios and based their analysis on the same case files. However, Brackmann and colleagues showed that the assumptions made by the second expert contradicted the scientific literature.

The above-discussed studies demonstrate that not all experts are knowledgeable about the scientific research literature. In contrast, Otgaar et al. (2017) discuss a Dutch case in which two independent psychological experts used scientific evidence appropriately. The case involved twenty children who reported abuse by two teachers at their elementary school. Both psychological experts noted several factors that could have jeopardized the validity of the children’s testimonies and reached similar conclusions based on their analyses.

New Zealand

Zajac et al. (2013) reviewed expert psychological testimony provided in child sexual abuse cases in New Zealand. They identified three important misconceptions held by experts, concluding that ‘much of the expert psychological testimony presented in New Zealand courtrooms does not accurately or fairly represent the scientific literature’ (p. 615). For example, some experts testified that children generally disclose abuse increasingly over time and that a detailed testimony is a sign of accuracy, yet there is no scientific evidence to back these claims.

Practitioners’ Assessment of Expert Testimony

The gravity of the findings that many psychological expert evaluations across the world are of poor quality, depends on the extent to which the receivers of expert witness testimony (specifically judges, juries, attorneys, and prosecutors), are able to assess the quality of these reports and distinguish between high- and low-quality testimony. Findings in other forensic domains, such as DNA evidence, show that legal practitioners are generally unable to gauge the quality of expert testimony (e.g. De Keijser et al., 2016). If practitioners are similarly unable to detect poor-quality assessments by legal psychologists, such assessments carry a substantial risk of leading legal decision-makers to draw incorrect conclusions and, as a result, make wrong decisions, including innocence or guilt.

Findings of two studies investigating this question in the psychological domain are not encouraging. Chorn and Kovera (2019) found that the extent to which an intelligence test conducted by a psychological expert was reliable and valid, had little to no impact on judgments made by judges, attorneys, and mock jurors about the scientific quality of the test and the admissibility of the evidence. Similarly, in their review of forensic court reports in Australia, Goodman-Delahunty and Dhami (2013) noted a disconnect between empirical findings on report quality and perceptions of legal professionals. Whereas empirical findings revealed poor-quality forensic psychological reports (Doyle et al., 2011), survey findings showed that legal professionals were generally quite satisfied with the quality of forensic psychological reports (Day et al., 2000). This suggests that legal professionals may not be able to adequately determine the quality of expert assessments in the psychological domain.

Cognitive biases in legal psychological assessments

Human evaluations are notoriously influenced by cognitive biases (e.g. Gilovich et al., 2002). Even when we try to reach objective conclusions, we cannot help but be affected by our own expectations, beliefs, emotions, motivations, irrelevant contextual information, and others’ opinions. We generally use heuristics or shortcuts for making decisions in complex situations. Heuristics are simple and efficient thinking strategies that can be helpful in decision-making, because they reduce the required effort and time associated with a task (Shah & Oppenheimer, 2008). For example, deciding which yoghurt to buy based on which label looks the most appealing, rather than conducting a thorough investigation of nutritional values and flavour tests, may be beneficial to daily well-being. However, heuristics can also result in severe and systematic errors, known as cognitive biases (Tversky & Kahneman, 1974).

Research shows that experts in the forensic sciences (i.e. sciences concerned with the analysis of forensic evidence) are by no means immune from cognitive biases (Dror, 2018, 2020; Dror & Charlton, 2006; Kahneman et al., 2021). One area in which cognitive biases have been investigated extensively is that of pattern recognition. Many forensic disciplines involve a form of pattern recognition – that is, inspecting two samples and determining whether they are a match, such as evidence involving fingerprints, shoe prints, bitemarks, handwriting, or firearms. Studies on different forms of pattern recognition have reached similar conclusions. In contrast to what many people, including experts, may believe, pattern recognition involving forensic latent evidence from crime scenes is not a purely technical or objective task (for reviews, see Cooper & Meterko, 2019; Dror & Cole, 2010; Kukucka & Dror, 2022). Instead, these forensic domains are heavily influenced by the same factors that affect human evaluations in general. For example, forensic experts’ assessments have been found to be affected by colleagues’ previous assessments, the beliefs of the detective on the case, and information that the suspect has confessed or, conversely, has an alibi. That ‘potential for bias and error in human observers’ (p. 8) was highlighted in an influential report on the status of forensic science in the United States published by the National Academy of Sciences (2009).

Experts in other forensic domains, who do not compare patterns but conduct evaluations and assessments, have similarly been shown to be impacted by bias. For example, in Dror et al.’s (2021) study on forensic pathology decisions, a dataset of over 1,000 death certificates and a data set comparing decisions by 133 practitioners revealed that determinations of the victim’s manner of death (accident vs. homicide) were susceptible to bias by non-medical irrelevant information. Even in the domain that is traditionally viewed as the ‘gold standard in forensic science’ (Lynch, 2003, p. 93), namely DNA profiling, research has shown that interpretations of the evidence are significantly affected by domain-irrelevant information, such as whether the suspect has confessed or has been implicated by another suspect (De Keijser et al., 2016; Dror & Hampikian, 2011).

Research on cognitive biases in the forensic psychology domain was recently summarized in a systematic review by Neal et al. (2022). They conclude that ‘relatively few empirical studies have investigated biases and ways to reduce them in forensic mental health’ (p. 9). They found 17 studies that investigated cognitive biases, of which 58.8% found significant effects, 23.5% found partial effects, and 17.6% found no effects. More specific findings from their review will be discussed in the next section, in which we discuss how cognitive biases may affect legal psychological assessments. Discussing all of the many cognitive biases that exist would be beyond the scope of the current article, but we will highlight some cognitive biases that could particularly affect the different stages of the legal psychologist’s evaluation.

Potential Biases Before Seeing the Evidence

Before the legal psychologist’s evaluation has even started, numerous non-diagnostic factors may influence the expert’s expectations regarding the evidence. For example, findings on the criminal stereotypes bias (Smalarz et al., 2016) show that certain crimes (e.g. child sexual abuse) are associated with specific criminal stereotypes. In Smalarz and colleagues’ study, students first read a mock police report about either a ‘stereotyped’ crime (i.e. child molestation) or a ‘nonstereotyped’ crime (i.e. identity theft) and then judged whether a fingerprint found at the crime scene matched that of the suspect. In addition, participants were informed either that the suspect was a white male or an Asian female. For the nonstereotyped crime, whether the suspect was a white male or Asian female did not affect fingerprint judgments. In contrast, participants who read that a white male was suspected of molesting a child (i.e. consistent with the prevailing stereotype) were more likely to incorrectly judge that the two fingerprints matched than participants who read that an Asian female was suspected of child molestation. In a similar vein, legal psychological experts might be more likely to judge a child’s statement as valid or a line-up as fair if suspects fit the stereotype of the crime of which they are accused.

The familiarity of a case can also bias experts’ decision-making. In a study by Searston et al. (2016), psychology students first read 18 case reports, judged whether pairs of fingerprints matched and immediately received feedback on their matching decision. Then, participants read highly similar case reports and judged 18 new fingerprint pairs. For each case report, if the fingerprints had previously matched, now they did not match, and vice versa. The accuracy with which participants judged the new fingerprints decreased compared to the previous prints. Thus, a previous decision made in a similar case may lead an expert to mistakenly draw the same conclusion in their current case. The same could apply to legal psychologists, who are likely to be appointed as an expert in cases that share similar characteristics (e.g. an alleged victim who claims that while in therapy she recovered memories of childhood sexual abuse).

Furthermore, experts may be biased to favor the appointing party, which is commonly referred to as allegiance bias or adversarial allegiance (Merckelbach, 2016; Murrie et al., 2013; Neal et al., 2022). Thus, an expert appointed by the prosecution will likely draw more incriminating conclusions than an expert appointed by the defense. Experts who change their conclusions intentionally to benefit the appointing party are known as ‘hired guns’ (e.g. Saks, 1990, p. 296), but even experts who do not do so intentionally, may fall prey to allegiance bias. That could happen, for example, because they are exposed to the appointing party’s views, because they receive irrelevant contextual information that favors the appointing party, or because they like the appointing party more as a result of spending more time with them (see Murrie & Boccaccini, 2015, for a review of mechanisms involved in allegiance bias).

Evidence for allegiance bias in clinical forensic psychological expert testimony has been found both in field studies (Edens et al., 2015; Lloyd et al., 2010; Murrie et al., 2013) and experimental studies (McAuliff & Arter, 2016; Murrie et al., 2013). In their systematic review, Neal et al. (2022) report that out of the six studies that investigated adversarial allegiance, four found support and two found partial support for allegiance bias among forensic mental health experts. Allegiance bias has also been demonstrated in the context of legal psychological reports (Sauerland et al., 2020), although this study was conducted with students taking a course on legal psychology rather than actual legal psychological experts. The students were asked to act as expert witnesses on a child sexual abuse case, by answering several questions about the validity of testimony from a child claiming to have been abused. Participants who had received an appointment letter from the prosecutor deemed the incriminating testimony significantly more valid than participants who had been appointed by the defense. An important direction for future research is to investigate whether legal psychological experts also fall prey to allegiance bias.

Potential Biases After Seeing (Part of) the Evidence

Exposure to irrelevant contextual information about the case, such as that the suspect has confessed, molds experts’ expectations before they have even seen the to-be-evaluated evidence (e.g. Dror & Cole, 2010). These initially formed expectations guide subsequent evaluations of the evidence. Probably the most well-known cognitive bias is confirmation bias: the tendency to seek information that confirms previous beliefs (see Kassin et al., 2013, for a review of confirmation bias in forensic settings). That typically goes hand in hand with belief perseverance: the tendency to disregard or downplay information that disconfirms previous beliefs (Ross et al., 1975; see Jelalian & Miller, 1984, for a review). These tendencies can be explained by cognitive dissonance theory (cf. Elliot & Devine, 1994): When people encounter information that calls into question their preexisting beliefs, it creates an uncomfortable feeling of tension, known as cognitive dissonance. To reduce the dissonance, people tend to explain away the contradictory information and find more evidence to support their original beliefs. In the forensic psychology domain, Griffith (2019) investigated confirmation bias among mental health professionals. Participants read a case vignette and were asked to select one of two hypotheses about the case. Next, they indicated which information they would want to receive to test the selected hypothesis. Participants significantly preferred to receive confirmatory information over disconfirmatory information, providing support for confirmation bias.

Let us consider a concrete example of how confirmation bias and belief perseverance could influence a legal psychological assessment. Imagine that a legal psychological expert discovers that DNA material on the victim did not belong to the suspect (e.g. because the lawyer has told them or because the expert read it in the case file) and forms an initial belief that the suspect is innocent. In the subsequent evaluation of eyewitness statements, the expert is then likely to focus on elements that support the suspect’s innocence, and downplay the importance of elements that support the suspect’s guilt. In addition, the initial belief that the suspect is innocent will affect the overall interpretation of the evidence (Charman et al., 2016; 2017), for instance, judging an incriminating eyewitness statement as less trustworthy or a line-up as poorly conducted. Finally, the expert’s initial belief in combination with their biased interpretation of the eyewitness statements further strengthens their belief that the suspect is innocent (Charman et al., 2017), which is likely to affect their final conclusion. This positive feedback loop has been described as the bias snowball and bias cascade effects (Dror, 2018; 2020; see also Charman et al., 2017).

Awareness of Cognitive Biases

To what extent are legal psychologists aware that cognitive biases can affect their evaluations? More generally, people tend to be unaware of their own susceptibility to biases, despite knowing about the bias in theory and being able to recognize it in others (Pronin & Kugler, 2007; Pronin et al., 2002). Forensic experts also fall prey to this so-called bias blind spot (Kukucka et al., 2017), including clinical forensic psychologists (Neal & Brodsky, 2016; Zapf et al., 2018). Neal et al. (2022) systematic review shows that three studies addressed the bias blind spot in forensic mental health professionals. Support for the bias blind spot was found in two studies, and partial support in one study. A possible explanation may be that people believe that they have insight into their own mental state, but consider other people’s introspections to be untrustworthy (Pronin & Kugler, 2007).

Although the above-mentioned studies show that both lay people and forensic experts fall prey to the bias blind spot, it is unknown whether legal psychologists also evince a bias blind spot. Given that cognitive biases are the bread and butter of legal psychology, and that legal psychologists frequently report on the many ways in which different players in the investigative process may have been biased and may have failed to see their own biases, one could hypothesize that legal psychologists are at least more aware of potential cognitive biases in their own work than other forensic experts. However, at present, empirical research on this issue is absent. It may well be that legal psychologists fall prey to the same pitfalls about which they always warn others. But even if they are more aware of their own cognitive biases than the average person, the question is to what extent they can successfully guard against them. This will be discussed in the next section.

Reducing cognitive biases

Many anti-bias interventions (not only in legal psychology) aim to reduce bias by raising awareness. Although awareness is important, as it enables implementation of bias reducing measures, awareness on its own is not effective. People often have the ‘illusion of control’ (see Dror, 2020), believing they can control their thought processes and biases by mere willpower. However, actual measures must be implemented to effectively minimize bias.

Blind Procedures

One of the most effective, evidence-based methods of reducing the potential impact of cognitive biases is the use of blind procedures (Robertson & Kesselheim, 2016). Blind procedures restrict the information presented to give the expert what they actually need for their evaluation (i.e. task-relevant information). For example, a forensic expert who assesses fingerprints should not be exposed to information about suspect confessions or previous criminal convictions. Context management procedures can be used to control what is presented to the expert, and at what stage of the evaluation (Dror, 2020).

In forensic domains in which examiners compare patterns, such as fingerprint, firearms, or handwriting, one must make sure that the examiner is working from the evidence to the suspect, not backwards from the suspect to the evidence. That can be achieved by controlling when information is presented to the experts: The presentation should always start with the actual evidence (e.g. from the crime scene), and then linearly and sequentially unmask further information, for example the fingerprint or DNA profile of the suspect (see Linear Sequential Unmasking, LSU; Dror et al., 2015). Furthermore, LSU-Expanded prioritizes the evidence sequence based on its level of objectivity and relevance (Dror & Kukucka, 2021).

Another approach is to have an evidence line-up, whereby experts are provided with a number of potential samples to be compared and matched, without knowing which sample belongs to the suspect (Kukucka et al., 2020; Smith et al., 2020). Such procedures are not unfamiliar to legal psychologists, who typically recommend line-up rather than show-up procedures in the context of eyewitness identifications, because show-ups produce more false identifications than line-ups (Clark, 2012; Steblay et al., 2003). Also, legal psychologists advise that line-ups are administered double-blind; that is, the person who shows the line-up to an eyewitness should not know which line-up member is the suspect (Charman & Quiroz, 2016; Greathouse & Kovera, 2009; see also the scientific review paper on eyewitness identification recommendations; Wells et al., 2020).

For some types of evaluations that legal psychologists are asked to perform, blind procedures might be implemented to avoid the risk of cognitive bias. For example, to evaluate the quality of an eyewitness identification line-up and how it was administered, an expert does not need to know of what crime the suspect is accused, or what other evidence there is against the suspect. Yet, based on our experience, that type of information is usually provided to the legal psychological expert even when the assignment concerns solely the quality of the line-up. Ensuring that the case file provided to the expert consists exclusively of the materials relevant to the legal psychological assessment would reduce the risk of cognitive biases affecting the expert’s analysis. However, it should be noted that the appointing party is unlikely to be able to determine what constitutes relevant and irrelevant information for the legal psychological expert. This is nicely illustrated by a case in which Conway served as an expert:

the fact that she had been in “survivor” counselling for 3 years prior to making her witness statement, and lived with a woman during that period who had written a book on survivors of childhood sexual abuse, was apparently considered irrelevant to the “false memory” defence with its focus on the escalating memories of abuse (Conway, 2013, p. 570)

Even when the expert is given a narrow assignment, such as evaluating the quality of a line-up, there may be information elsewhere in the case file that is highly relevant to the legal psychological assessment, such as an eyewitness’s comment (e.g. in an investigative interview or surreptitiously recorded conversation) that she had seen the perpetrator on another occasion. We therefore recommend that the selection of materials to be provided to the expert is made by a case manager who is knowledgeable on the materials needed for legal psychological analysis, yet has no involvement in the case (Dror, 2020). That would require some organization (e.g. a small additional fee billed by the case manager for the time spent making the selection) but is by no means impossible to achieve. This method has for example been used for a long time already in voice identification expert work in The Netherlands (Broeders, 1996, 2003).

When Blind Procedures Are Not Feasible

Even though some types of specific assignments posed to legal psychologists could be suitable for blind procedures, such as evaluating the quality of a line-up, we would argue that much of the work is not. In many cases, legal psychologists are asked to evaluate the validity of testimonies provided by suspects, eyewitnesses, and victims. Contrary to widespread beliefs (e.g. as shown on popular TV shows such as Lie to Me) and commonly used police interrogation trainings (e.g. the Reid Technique; Inbau et al., 2013), scientific evidence clearly shows that it is impossible to judge whether a statement is trustworthy based solely on the characteristics of the statement itself, or the behavior of the person who is providing the statement. Specifically, decades of research on non-verbal lie detection show that people perform barely above chance (around 54% accuracy) in determining whether someone is lying or telling the truth, and that experts fare no better than laypeople (for a meta-analysis, see Bond & DePaulo, 2006). Moreover, Hartwig and Bond’s (2011) lens model meta-analysis showed that observers perform so poorly on lie detection not because they are paying attention to the wrong cues, but rather because there are simply very few behavioral cues that distinguish between liars and truth-tellers. Thus, one cannot tell by looking at a person whether they are lying or not.

Verbal lie detection tools such as Reality Monitoring and Statement Validity Analysis fare only slightly better (Vrij, 2015) and scoring statements with such tools can be affected by contextual biases as well (Bogaard et al., 2014). Even though somewhat higher accuracy percentages have been reported for these tools — around 70% (Oberlader et al., 2016) — most experiments in which the tools were tested used designs far removed from the circumstances of real eyewitnesses or suspects in criminal cases. Moreover, even if the experimental designs had higher ecological validity, an accuracy percentage of 70% is not sufficient to draw any strong conclusions about a particular statement made by a particular person in a particular case, especially when considering legal standards such as ‘beyond a reasonable doubt’ (see e.g. Newman, 2006; Tillers & Gottfried, 2006; for comments on quantifying the ‘reasonable doubt’ standard).

Because it is often impossible to effectively evaluate the validity of statements based on the content or presentation of the statements alone, an evaluation of statements requires contextual information. It is therefore not feasible to implement blind procedures across the board for legal psychological assessments, and we need to think more deeply about which contextual information should and should not be used, as well as other measures to reduce cognitive biases.

The Alternative Scenario Method

One method proposed by legal psychologists in The Netherlands to reduce the impact of cognitive biases is the alternative scenario method. Although this method was introduced already three decades ago in the Dutch legal psychological literature (Crombag et al., 1992; see also Crombag & Wagenaar, 2000; Rassin, 2001, 2014; Van Koppen, 2017, 2022), it has only recently gained traction in the international literature (Otgaar et al., 2020; Otgaar et al., 2017; Rassin, 2018; Van Koppen & Mackor, 2020). The gist of the alternative scenario method is not much different from how evidence is evaluated in most areas of scientific endeavors (see Popper, 1963; Popper, 1980; for a more elaborate explanation, see Van Koppen & Mackor, 2020). In its simplest form, the expert must explicitly formulate at least two scenarios to evaluate the evidence. For instance, ‘the result of the line-up is valid’ and ‘the result of the line-up is not valid’; or ‘the eyewitness statement about a particular issue is based on a genuine memory’ and ‘the eyewitness statement about that issue is not based on a genuine memory’. After the expert has discussed the evidence, an explicit discussion should follow in which the expert argues to what extent the findings discriminate between the scenarios (or are best predicted by the scenarios).

To illustrate the alternative scenario method, consider the following concrete example. A legal psychologist is asked to evaluate the validity of testimonies of an alleged victim who claims to have been sexually abused by her grandparents. When reading the case file, the legal psychologist discovers that the ‘memories’ of the alleged victim were recovered during suggestive therapy and were absent before the therapy. In this situation, a legal psychologist might conclude that there is little evidence to support the scenario that the testimonies are valid, whereas there are indications that the testimonies are not valid (i.e. suggestive therapy).

There is some tentative evidence that thinking in alternative scenarios might be a promising strategy in expert witness work. Specifically, in O’Brien’s study (2009, Experiment 2), students read a mock case file of a criminal investigation concerning a home invasion and shooting. After reading approximately half of the case file, one group of participants was asked who they thought had committed the crime (hypothesis condition). Especially relevant for the current purposes was another experimental group that was asked who they thought had committed the crime and why, but also why that person may be innocent (counterhypothesis condition). These experimental groups were compared with a control group who simply continued to read (no-hypothesis condition). After participants finished reading the file, they answered questions about the criminal investigation, including recall of case information and which lines of investigation to pursue. O’Brien found that participants in the hypothesis condition recalled significantly more information in line with the suspect’s guilt and advised significantly more lines of investigation focused on the suspect than participants in the no-hypothesis and counterhypothesis conditions (which did not differ from each other). Thus, when an expert reading the case file forms initial expectations about a particular suspect’s guilt, the expert may be able to reduce their bias toward that suspect by explicitly formulating why the suspect may be innocent (i.e. generate an alternative hypothesis).

Although Neal et al. (2022) systematic review of forensic psychological literature revealed a scarcity of research on debiasing techniques, one such technique showed promise: the ‘consider-the-opposite’ strategy (see also Mussweiler et al., 2000). This technique involves experts asking themselves why their initial judgment might be wrong and considering possible alternatives, just like the alternative-scenario method. In Griffith’s (2019) case vignette experiment, forensic mental health professionals showed confirmation bias overall, but this was reduced when professionals viewed an alternative hypothesis and were asked to list reasons why this hypothesis could be correct.

Additionally, it has been shown that jurors have a more critical attitude towards eyewitness evidence when they are asked to think about alternative outcomes. For example, Rodriguez and Berry (2016) gave mock jurors a summary of a crime consisting of high-quality evidence (e.g. unbiased line-up instructions) or low-quality evidence (e.g. biased line-up instructions). Some participants were asked to engage in a counterfactual thinking style (i.e. thinking about how the situation could have been different), which bears resemblance with coming up with an alternative scenario. The other half were not instructed to use this thinking style. The most striking finding was that participants who had been provided with low-quality eyewitness evidence were least likely to state that the suspect was guilty if they had engaged in a counterfactual mindset.

In contrast to this work, in Sauerland et al.’s (2020) study with students as participants, a significant effect of alternative-scenario instructions was not found. After participants in their Experiments 2 and 3 had read the case file, half received instructions to consider alternative scenarios, while the other half did not. As mentioned earlier, an allegiance bias was observed in that participants appointed by the prosecutor deemed the incriminating testimony as more valid than participants appointed by the defense. This allegiance bias was not reduced by instructions to consider alternative scenarios. A potential explanation for the ineffectiveness of the instructions in this study is that participants received the instructions after reading the case file. Perhaps, allegiance bias would have been reduced if participants had been instructed to actively consider alternative scenarios before they had read the case file. Regardless of when the instructions are given, the success of considering alternative scenarios lies in a true and sincere attempt to really consider each scenario as viable, rather than a ‘box-ticking’ exercise.

Other Bias Correction Strategies

Other potential safeguards could be borrowed from related fields of expertise, such as the bias correction strategies identified by Neal and Brodsky (2016) in the context of forensic mental health evaluations. These strategies include measures we have already mentioned, such as blind procedures and considering alternative scenarios, but also other strategies that could be relevant for legal psychologists. For example, legal psychologists could be advised to participate in continued professional development (e.g. courses, academic conferences) to update and refresh their knowledge on cognitive biases. This is in fact a requirement for registration as a court expert in some countries (e.g. Netherlands Register of Court Experts, 2020). Legal psychologists could also adopt the ‘slowing down’ strategy recommended by Neal and Brodsky: taking time to think about the analysis instead of writing it down immediately may be a useful technique for reducing bias (see also Croskerry et al., 2013; Moulton et al., 2010). Just like mental health experts, legal psychologists should use evidence-based structured evaluation methods where appropriate and available, such as in the context of malingering assessments (cf. Shura et al., 2022; Van Impelen et al., 2014). Two other debiasing strategies suggested by forensic clinicians in Neal and Brodsky’s study, which could be promising but are not yet supported by empirical evidence, are the recommendation to take careful notes during the analysis (which can be used to generate alternative hypotheses after initial review of the evidence) and to develop a sense of pride in one’s professional identity (which can motivate experts to avoid negative perceptions of being a ‘hired gun’).

Goldyne (2007) also proposed a set of introspective tasks intended to detect and minimize bias proactively. Specifically, the tasks are meant to cue the expert’s recognition of emotional and non-emotional factors that could underlie their own cognitive biases. However, the effectiveness of the tasks has not been tested empirically. Given that people are often unaware of their own biases, it seems unlikely that introspection would reduce bias (see also Neal et al., 2022, who make a similar point).

A final potential safeguard is to ask an expert colleague to carefully read and provide feedback on the expert witness report before it is sent to the commissioning party (see also Otgaar et al., 2017). The task of the colleague is to critically examine whether the expert’s conclusions are appropriately supported by the scientific literature and the case information.³ Furthermore, the colleague could adopt the role of a devil’s advocate by critically searching for alternative scenarios, thereby potentially reducing bias. Devil’s advocacy has robustly been shown to improve decision-making in other contexts (for a meta-analysis, see Schwenk, 1990). The peer review process can help experts identify potential blind spots, problematic reasoning, or partisan tendencies in their writing (see the code of conduct published by the Netherlands Register of Court Experts, 2015, for similar recommendations). Critical for the success of this approach is that the expert colleague is not exposed to irrelevant contextual information and is given only the relevant data that should underpin the report. Evidence that a peer review requirement can work in practice comes from observations in The Netherlands, where legal psychologists now ask colleagues to peer review their reports as a matter of course (free of charge, similar to academic peer review), in line with the code of conduct published by the Netherlands Register of Court Experts (2015).

Guidelines for legal psychological experts

The great variability in the quality, format, content, and conclusions of expert witness reports in the legal psychological domain could be reduced by adopting evidence-based guidelines. Various sets of guidelines for forensic psychology experts have been proposed (e.g. American Psychological Association, 2013; Committee on Ethical Guidelines for Forensic Psychologists, 1991; Conroy, 2006; Weiner, 2013), but these tend to focus predominantly on procedural issues surrounding expert testimony or clinical forensic evaluations, rather than how to conduct high-quality legal psychological analyses in the context of expert reports. For example, the American Psychological Association (APA) put forward several specialty guidelines for forensic psychologists such as being impartial and determining fees. However, no specific guidelines were established that might mitigate the pernicious effect of biases on expert witness work. One guideline that does show some resemblance with the alternative-scenario method is APA’s guideline on the use of appropriate methods during forensic psychological work. In this specific guideline, it is postulated that ‘forensic practitioners seek to maintain integrity by examining the issue or problem at hand from all reasonable perspectives and seek information that will differentially test plausible rival hypotheses’ (pp. 14-15). The recommendation that rival hypotheses should be tested is in line with the idea of testing different scenarios.

Otgaar et al. (2017) proposed three general recommendations for expert reports about the validity of statements, and Cutler and Kovera (2010) provide advice for expert reports concerning eyewitness identifications, but to our knowledge, we are the first to propose a set of detailed guidelines for the content of legal psychological expert witness reports in general. The present guidelines are designed to promote well-written reports, free of jargon, that contain analyses based on evidence, in which scientific insights are clearly linked to the case at hand, and measures are taken to reduce cognitive biases. We hope that the process of structuring the report and adhering to these guidelines will also impact how legal psychologists carry out their assessments. In other words, the manner in which a report is written has constituting powers that influence the actual work—similar to language, which does not only reflect thoughts, but also plays a constitutive role in their formation.

The guidelines are based on the research findings described in this article, supplemented by insights gained during a panel meeting of ten legal psychologists who serve as expert witnesses in The Netherlands (Vredeveldt et al., 2017). Even though the guidelines were initially developed in the context of the Dutch inquisitorial system, they are designed to be broadly relevant to legal psychologists around the world. Nonetheless, it is worth noting that courts in certain jurisdictions may have different expectations of reports, and that dealing with these differences can be challenging. We recommend that all elements listed below are incorporated into legal psychological expert witness reports, unless there is a good reason to deviate from the guidelines.

Assignment

A legal psychological report should start with a clear and precise description of the received assignment (i.e. instructions), including who appointed the expert. If the received assignment was unclear, the expert should contact the appointing party for additional clarification. If the expert cannot answer one or more of the posed questions, the report should state why that question could not be answered. In sum, the expert witness report should include the original assignment and question(s), how the expert interpreted the assignment and, if applicable, which questions were unanswerable and why. Especially critical is to include any ‘briefing’ and ‘background information’ that was given, and any ‘expectations’ of what the evaluation should look like.

To achieve this, all relevant communications (e.g. phone, e-mail) with the appointing party (or any other parties involved in the case) must be carefully documented. Especially when provided information does not appear directly relevant to the assignment, it is crucial that the expert records exactly what information they received and when, because irrelevant contextual information can affect evaluations concerning the case. Such influences cannot always be completely avoided, but the expert should at least be as transparent as possible about the potential sources of influence.

Received Materials

The report should include a list of all received materials (i.e. full disclosure of what information was known; see Almazrouei et al., 2019). If the expert has not consulted all received materials for the report, a distinction should be made between consulted and non-consulted sources. If the expert believes they have not received all relevant materials, they should contact the appointing party to check whether these materials are available. For instance, if an eyewitness interview was audio- or video-recorded but the recording was not provided, the expert should insist on receiving the recording before they conduct the analysis. If the appointing party will not or cannot provide materials essential to the analysis, the expert can decline the assignment, or, at the very least, the expert should note in the report that relevant materials are missing and discuss how these omissions have limited their conclusions. For example, if the expert’s analysis is limited to an interview transcript because the recording of an eyewitness interview was lost, the expert should note that the transcript may not accurately reflect what was said, and that information about the non-verbal behavior of the eyewitness and the police interviewer was missing. Another example is described by Conway (2013), who acted as an expert witness in a sexual abuse case in which the alleged victim had undergone therapy, but no notes had been taken during the therapy sessions. As a result, Conway was unable to assess the influence of the therapy on the alleged victim’s memories of childhood abuse.

Expertise

The expert should elucidate their expertise on the topic of the assignment. They can refer to their relevant scientific publications, previous reports in similar cases, and attach a (shortened) biography or curriculum vitae to the report. In addition, if a relevant code of conduct or expert register exists in the country in which the expert is practicing, the expert should state whether they adhere to that code of conduct and whether they are registered as a court expert. If a report is written collaboratively, the report should specify the contributions and expertise of each expert.

Context Summary

Before describing the approach and analysis, it is useful to summarize the elements of the case that are relevant to the analysis. For example, the expert may describe what the case is about, who are involved, and when important events took place (e.g. investigative interviews, eyewitness identifications, court hearings). The context summary serves to aid the expert’s memory if they are asked to clarify the analysis some months or even years later. Moreover, it informs the fact finder about the impression the expert has formed about the case. If a context management procedure was implemented to limit the impact of irrelevant contextual information, then the expert first conducts the analysis and writes down the findings, before receiving information about the context of the case. Once that information has been received, the expert should assess whether it contains any details that are relevant for answering the question at hand. During the final stages of writing the report, the expert can add the context summary.

Approach and Analysis

The expert should explain which approach or method they used. That explanation can either be described in a separate section prior to the analysis or integrated into the analysis. Both the approach and the analysis must be based on scientific literature. As explained above, we recommend that, when possible, legal psychologists use alternative scenarios or hypotheses to analyze the evidence (Otgaar et al., 2017; Rassin, 2018; Van Koppen & Mackor, 2020). The expert may start with two main scenarios. For example, the first scenario could be that an eyewitness statement is based on a genuine memory and the second scenario could be that the statement is fabricated. During the analysis, additional scenarios may emerge. For example, a third scenario could be that parts of the eyewitness statement are based on a genuine memory, but other parts of the testimony are fabricated.

If the expert uses any diagnostic tools, they should describe why that tool helps to answer the question(s) and what the outcomes and interpretations of the tool are in the case at hand. The expert should also explain the scope and limitations of the tool and its interpretation. Where applicable, the expert should describe the psychometric specifications, duration of the test, and formal evaluations or approvals related to the tool.

If the analysis concerns testimonies from suspects, eyewitnesses, or victims from a different cultural background than that of the expert, the expert should reflect on the extent to which cross-cultural differences may have influenced the evaluation of the testimonies. For example, research shows that statements from African asylum seekers and atrocity eyewitnesses may be less detailed and consistent than what is considered the norm in Western societies (e.g. Anders, 2011; Combs, 2017; Herlihy et al., 2012). Therefore, the expert should be careful not to draw erroneous conclusions based on their own culture-specific expectations of what a statement should look like.

Finally, it is important that the expert states which measures were taken to reduce cognitive biases, if any. For example, the report may specify the order of examination of the materials, and whether and how context management procedures were used to minimize bias. We recommend that experts ask another expert in their field to conduct a critical peer review of a draft version of the expert witness report, to obtain feedback on the content, the readability, and potential bias in the report (cf. Otgaar et al., 2017).

Conclusion

In the conclusion, the expert should summarize the main points of the analysis, explain how the evidence fits with the alternative scenarios, be transparent about any irrelevant contextual information to which they may have been exposed and how this could have biased their evaluation, and ultimately answer the question(s) posed by the appointing party. Crucially, experts should refrain from making a judgment about whether the suspect is guilty or not, since that question is outside of the experts’ purview and should be left to the fact finder. Sometimes the conclusions of a report may come close to a decision about guilt. For instance, if the expert reviews a child’s testimony that constitutes the major evidence in a sexual abuse case and concludes that the evidence supports the scenario that the child’s testimony is based on a genuine memory, that comes quite close to the conclusion that the defendant has committed the abuse. Particularly in these kinds of cases the expert should carefully conclude solely in terms of an evaluation of the child’s testimony.

The conclusion may be provided in the middle or at the end of the report. If the conclusion is presented in the middle (i.e. before the summary, approach, and analysis), it is recommended to summarize the main points of the analysis once more at the end.

Obiter Dictum

If at any point during the case analysis, the expert makes an important observation that is not covered by the assignment but is within their field of expertise, they should highlight this at the end of the report. For instance, if an expert is asked to evaluate a suspect’s testimony but notices in the case file that an eyewitness was interviewed in a highly suggestive manner, it is important that they make a note of this in the report. This gives the judge, defense, or prosecution an opportunity to pursue further investigation on this point.

Literature

The expert must refer to evidence from scientific studies in their analyses, using studies published in peer-reviewed journals or books. Where possible, meta-analyses and systematic literature reviews should be cited. The expert should also indicate to what extent the scientific findings have been replicated and how they apply to the case at hand.

If there is relevant information that does not need to be described in the report itself, such as the expert’s curriculum vitae or technical specifications of administered tests, the expert should include that information in appendices.

Concluding remarks

In the present article we described findings on the quality of psychological expert witness reports, potential cognitive biases to which the legal psychologist may fall prey, as well as possible safeguards to reduce bias and guidelines for writing legal psychological expert witness reports. To find out to what extent the cognitive biases that we have described here affect real expert witness reports, more empirical research is needed. Sauerland et al.’s (2020) experiment on allegiance bias among legal psychology students constitutes a first step in that direction, but we also need in-depth analyses of expert witness reports written by experienced legal psychologists in real cases. This seems particularly relevant in cases in which experts come to drastically different conclusions based on the same information (see e.g. Brackmann et al., 2016). An analysis of such disagreements would be of great interest not only from a scientific perspective, but also from a legal perspective. Its importance is illustrated by the recent call of the Attorney General of the Supreme Court of The Netherlands for ‘legal psychological research into the reliability (consistency) and validity of legal psychological research’ (Aben, 2021, p. 23, our translation). The irony of discussing how cognitive biases affect experts who report about cognitive biases, does not escape us. It is time for the experts on bias to shed some light on their own biases.

Declaration of interest statement

Some of the guidelines in this article were previously published in Dutch (Vredeveldt et al., 2017). We have no conflicts of interest to disclose. Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 The terms ‘reliability’ and ‘validity’ are sometimes used interchangeably, but they have different meanings. Reliability, in psychological terms, refers to the extent to which an observation is repeated, consistent and reproducible (Kahneman et al., 2021). Applied to eyewitness testimony, the question might be whether a witness tells the same story on separate occasions (also known as between-statement consistency, cf. Vredeveldt et al., 2014). Validity, in contrast, refers to the extent to which a statement accurately reflects what has happened in the past. Legal professionals often use these terms differently, but in this article, we will adhere to the psychological scientific definitions.

2 For example, Neal (2018) defined forensic psychology as “a subfield of psychology in which basic and applied psychological science or scientifically-oriented professional practice is applied to the law to help resolve legal, contractual, or administrative matters” (p. 652).

3 Note that this is different from a case manager who is hired to make a selection of relevant materials.