Exploring the predictive power of impulsivity measures in predicting self-reported and informant-reported inpatient disruptive behaviors

ABSTRACT

Impulsivity is strongly associated with aggression and antisocial conduct. Although self-report measures are a time-efficient means to assess impulsivity, they may be susceptible to socially desirable responding, particularly in forensic psychiatry. The current study aimed to investigate the predictive validity of three measures of impulsivity in predicting self- and informant-reported antisocial behavior: the Barratt Impulsiveness Scale, the Self-Centered Impulsivity scale of the Psychopathic Personality Inventory-Revised and the general Disinhibition factor of the Externalizing Spectrum Inventory. Next, the mediating role of a measure of self-deception, the Virtuous Responding scale, was examined in these associations. Participants (N = 94) were inpatients from addiction care and forensic psychiatry. Two regression analyses were conducted using self-reported antisocial behavior in the first, and informant-reported antisocial behavior in the second analysis as outcome variables. In addition, a mediated regression analysis was conducted, using the Virtuous Responding scale as a mediator. The impulsivity measures showed a substantially lower predictive validity when informant-reported behavior was predicted. The Virtuous Responding scale appeared to be unreliable in the current sample and showed no mediation effect. The results showed insufficient support for the predictive validity of the three measures of impulsivity. Alternative time-efficient assessments for impulsivity are needed, such as informant-based measures.

KEYWORDS:

• Impulsivity
• disruptive behaviors
• questionnaire
• predictive validity

Introduction

Antisocial and aggressive behaviors constitute a major problem in mental health care and can form a threat to the well-being of healthcare professionals (Pekurinen, 2017). Commonly, impulsivity is considered one of the main personality characteristics that is strongly associated with antisocial conduct, comorbid substance abuse, and aggression (Creswell et al., 2018; Dom et al., 2006). Furthermore, impulsivity has been found to underlie externalizing disorders such as Conduct Disorder and Antisocial Personality Disorder (Beauchaine & Saunder, 2017). Specifically, impulsivity has been related to observed inpatient physical aggression within the first 12 weeks of admission (incidence rate ratio 3.07) in a medium security forensic setting (Bousardt et al., 2016). Roozen et al. (2011) reported a moderate correlation (r = .42) between impulsivity and self-reported aggression in addiction care.

Impulsivity is measured through a variety of methods, such as neuroscience tasks, interviews, and self-report inventories. Of these methods, self-report measures are the more cost- and time-efficient assessment instruments. This may be the reason why many clinical approaches make use of self-report measures to assess impulsivity. Three self-report measures that target impulsivity are the Barratt impulsiveness scale (BIS; Patton et al., 1995), the PPI-R Self-Centered Impulsivity scale (Lilienfeld & Widows, 2005), and the Externalizing factor of the Externalizing Spectrum Inventory (ESI; Krueger et al., 2007), of which the BIS is a well-established and the most widely used self-report measure of impulsivity. For instance, Umut et al. (2017) reported a significant association between the BIS and the presence of interview-based antisocial personality disorder (ANCOVA: F = 5.285; p < .05) in a patient sample with heroin use disorder. Fields et al. (2015) reported a significant but low correlation (r = .15) between the BIS and the number of observed violent incidences in a sample of forensic inpatients, prisoners and patients in compulsory substance abuse treatment. The PPI-R Self-Centered Impulsivity scale (SI) and the ESI externalizing factor (Ext) have been researched in forensic and addiction care samples as well, but are less commonly used as measures of impulsivity. SI measures impulsivity as a subdomain of the psychopathic personality, while in the Ext, impulsivity is assumed to be the underlying factor of all externalizing traits in a bifactor model. SI has been associated with both self-reported reactive (r = .57; Cima & Raine, 2009; r = .40; Long et al., 2014) and self-reported proactive aggression (r = .60; Cima & Raine, 2009; r = .40; Long et al., 2014). Impulsivity as measured by the Ext, has been shown to correlate with self-reported rule-breaking behavior in psychology students (r = .75; Hall et al., 2007) and with interview-based adult antisocial behavior in prisoners (r = .45; Venables & Patrick, 2012).

Although an association between the BIS, SI, Ext and antisocial conduct has been reported in multiple studies, evidence of the association between the aforementioned impulsivity measures and informant-reported behaviors is lacking. Moreover, the use of self-report measures in forensic psychiatry and addiction care has been widely criticized. A common criticism is that results from self-report measures are unreliable, because forensic and addicted patients tend to give socially desirable answers (e.g. Achenbach, 2006). Often, social desirability scales are added to self-report measures to assess the risk of bias, such as the Virtuous Responding scale to the PPI-R (VR; Lilienfeld & Widows, 2005). Social desirability is usually divided into impression management and self-deception (Paulhus, 1984). Impression management is assumed to be motivated by deliberate misleading, while self-deception is assumed to stem from a lack of awareness of one’s personal weaknesses. The VR scale is assumed to measure self-deception (Uzieblo et al., 2010). Self-deception has been associated with personality disorders and relapse in substance abuse (De La Villa Moral Jiménez & Sirvent Ruiz, 2014). However, the use of social desirability and self-deception scales is also subject of debate. For instance, McGrath et al. (2010) concluded in their review that there is insufficient evidence to support the use of validity scales, and questioned whether any known scale can accurately assess self-deception or impression management. Hildebrand et al. (2018) reported in their meta-analysis a significant but low (r = −.12) effect size for the association between self-deception (which was encoded by the authors by means of the Balanced Inventory of Desirable Responding inventory; Paulhus, 1984) and self-reported risk assessment in offender samples. The role of the VR scale in the association between self-reported impulsivity and antisocial behavior has not yet been investigated.

The current study

Based on the findings of previous studies, we may assume that self-reported impulsivity shows a sufficient predictive validity when predicting self-reported antisocial behavior. However, the predictive power when predicting informant-reported behavior is unclear. In addition, the effect of self-deception, particularly the VR scale, on the predictive validity of self-reported impulsivity is unknown. The current study aims to explore this research caveat, by investigating whether the impulsivity scales of the BIS, SI and the Ext predict informant-reported antisocial behavior equally well as these measures predict self-reported antisocial behavior. This study will be conducted using a sample of inpatients in addiction care and medium security forensic psychiatry. Firstly, we will investigate the predictive validity of the three impulsivity measures combined, using self-reported and informant-reported inpatient antisocial behaviors as criterion variables. Secondly, we will examine the incremental validity of each impulsivity measure. Thirdly, we will analyze the mediation effect of the VR-scale (self-deception) in the relation between impulsivity and informant reported antisocial behaviors. We expect to find stronger support for the predictive validity of the aforementioned self-report impulsivity measures, when the self-reported measure of antisocial behavior is used as the criterion variable rather than the informant-reported version of this criterion variable (h1). Next, we expect that each separate measure of impulsivity (BSI, SI, and Ext) will contribute equally to the predictive validity (h2). Finally, we expect to find a significant mediating effect of the VR-scale on the predictive validity of all impulsivity assessments. Specifically, we expect that the predictive validity of the impulsivity measures will improve when the VR scale is added (h3).

Methods

Participants

The participants (N = 94) were inpatients from several forensic psychiatry and addiction care hospitals in The Netherlands. These patients were a subset of a larger dataset, consisting of clinical (n = 149) and nonclinical participants (n = 405). The data collection was approved by the medical ethical committee of the medical centre in Twente, The Netherlands (ID number METC/10,078.soe). The data were collected between January 2011 and November 2012. The original patient sample was representative of the Dutch forensic psychiatric population regarding age and gender (Ministry of Justice and Security, 2018, Wisseling et al., 2016). The patients were recruited at a medium security forensic hospital, six drugs rehab clinics and a forensic rehab clinic. Participation was voluntary. Patients who did not meet any exclusion criteria were recruited through a written informed consent letter. The exclusion criteria were: Psychotic disorders, severe brain damage, and illiteracy with regard to the Dutch language (Soe-Agnie et al., 2020). Patients were included in the current study if their main psychiatric nurse had filled out the informant version of the Adult Behavior Checklist, initially resulting in 97 enrolled patients. Three of these patients had to be excluded, due to not responding to the self-report measures. The demographic characteristics of the 94 patients are presented in Table 1. The gender ratio and the mean age of this sample were comparable to the characteristics found for the original sample (% females _n=149 = 15.6%; Mage _n=149 = 38.45, SD = 9.08). In addition, the gender ratio was comparable to the ratio found for the entire inpatient population in Dutch forensic and addiction care, which lies between 10% and 25% (“Custodial Institutions Agency”, 2019; Wisseling et al., 2016). The mean age of the female participants (M = 43.33; SD = 9.88) was significantly higher (t = −2.651; p < .05) than the mean age of the male participants (M = 36.57; SD = 8.84). The information on criminal records was self-reported. For this reason, we cross-checked the information provided by the patients by examining clinical files. We were able to retrieve file information on prior convictions of 37 patients. In this subgroup, the self-reports revealed a higher absolute number of patients with prior convictions than the file information (Table 1).

Table 1. Subject characteristics

	Total n = 94	Male n=78	Female n=15^1
Mean Age (SD)	37.65 (9.28)	36.57 (8.84)	43.33 (9.88)
Dutch ethnicity	78 (80.4)	64 (82.1)	14 (93.3)
DSM-IV classification
SUD only	2 (3.8)	0 (0.0)	2 (20.0)
Psychiatric disorder, no SUD	5 (9.6)	4 (10.3)	0 (0.0)
Dual diagnosis	39 (75.0)	31 (39.7)	6 (60.0)
SUD + delayed axis II disorder	6 (11.5)	4 (5.1)	2 (20.0)
Criminal record ^2
Convicted I%	72 (77.4)	66 (84.6)	6 (40.0)
Convicted II % (n=49)	46 (93.9)	36 (92.3)	8 (100.0)
Currently in forensic treatment	30 (57.7)	28 (70.0)	1 (6.7)

Note. SUD=Substance use disorder

¹The gender of one patient is missing. ²Criminal record I is self-reported, using the following item: “Have you ever been convicted?” Criminal record II is based on clinical file information of 39 male and 8 female patients

Instruments

The Dutch Externalizing Spectrum Inventory (ESI-NL; see Soe-Agnie et al., 2020) is a 160-item self-report inventory with a 4-point (scored 0–3) item scale. The ESI contains 23 subscales (also indicated as facets) assessing a range of expressions of externalizing proneness tapping into aggression, irresponsibility and deceitfulness, impulsivity/sensation-seeking, blame externalization, and substance use/abuse. Patrick et al. (2013) reported high Cronbach’s alpha values (≥.85) for the subscales in a sample of prisoners, and proposed a hierarchical model in which the subscales are assumed to be subsumed under one general factor of Disinhibition and two specific factors of Substance Abuse and Callous Aggression. Only the general Disinhibition factor was used for the analyses in the current study. Patrick et al. (2013) reported a good criterion validity for this general factor, when Multidimensional Personality Questionnaire (Tellegen & Waller, 2008) subscales assessing ‘aggression’ (r=.59) and ‘lack of control’ (r = .65) were used as criterion measures.

The Barrat Impulsiveness Scale-11 (BIS; Lilienfeld & Widows, 2005; Patton et al., 1995) is the 11th edition of a self-report 30-item questionnaire which assesses impulsivity. Items were rated on a 4-point scale ranging from 1 (rarely/never) to 4 (almost always/always). In the current study the total BIS-11 sum scores were used. Previous research has demonstrated good reliability (Cronbach’s alpha = .83 and test-retest reliability = .83) for the total score in a sample of 1,577 college students and healthy adults (Standford et al., 2009), and in a sample of 1,920 offenders and forensic patients (Cronbach’s alpha = .86). In the current sample, the Cronbach’s alpha estimate of the total score was α = .85.

The Psychopathic Personality Inventory – Revised (PPI-R; Lilienfeld & Widows, 2005; Uzieblo et al., 2010) is a self-report inventory designed to measure psychopathic personality traits from a dimensional perspective. While the gold standard for the assessment of psychopathy, the PCL-R (Hare et al., 1990), focuses on antisocial and criminal behavior, the PPI-R focuses on a broader spectrum of personality traits. The instrument comprises 154 items, and each item is answered on a 4-point scale: 1 (False), 2 (Mostly False), 3 (Mostly True), 4 (True). The PPI-R produces a total psychopathy score as well as scores on the Fearless Dominance (three subscales), Self-Centered Impulsivity (four subscales), and Cold-Heartedness (one subscale) factors. The PPI-R also contains three scales that are designed to measure response styles: Deviant Responding, Inconsistent Responding and Virtuous Responding. Anderson et al. (2012) reported that response bias was sufficiently detected via the PPI-R validity scales in an offender and student sample. In the current study, the Self-Centered Impulsivity (SI) and the Virtuous Responding (VR) scale were used. A VR T-score ≥ 65 is considered to be an indication of a lack of insight into one’s dysfunctions. Low scores (T ≤ 35) are considered to be indications of low self-esteem or a lack of insight into one’s dysfunctions (Uzieblo et al., 2010). Cronbach’s alpha estimates of the Dutch version of the PPI-R SI were α = .90 in forensic sample and α = .91 in an addiction care sample (Uzieblo et al., 2006). Cronbach’s alpha values for the VR scale were .70 in de forensic and .61 in the addiction care sample respectively. In the current study, Cronbach’s alpha values were low for the VR scale (α = .50) and good for the SI scale (α = .89).

The Adult Behavior Checklist informant and self-report version (ABCL/ASR; Achenbach & Rescorla, 2003). The ABCL is an informant-based questionnaire, while the ASR is the self-report equivalent of the ABCL. The Adult Behavior Checklist was developed to assess a broad spectrum of manifest externalizing and internalizing symptoms in adults. Both inventories contain 126 items rated on a 3-point scale, ranging from 0 (‘not true’) to 2 (‘very true/often true’). In the current study, the items pertained to the behaviors of the patients during the last month prior to the test administration. The items were distributed over eight subscales of which three subscales constitute the Externalizing profile: Aggressive Behavior (16 items), Rule-Breaking Behavior (13 items), and Intrusive Behavior (six items). For the current study, we used the sum scores of the Externalizing profile. Good reliability estimates of the ABCL and ASR Externalizing profiles were found in a mixed sample of students and patients, with test–retest correlations of r = .92 and r = .91, and Cronbach’s alpha estimates of α = .93 and α = .89, respectively. The cross-informant correlation between the ASR and ABCL was r = .44 (Achenbach & Rescorla, 2003). In the current study, the Cronbach’s alpha was α = .92 for the Externalizing factor of the informant version (ABCL), and .80 for the self-report version (ASR). The cross-informant correlation between the ABCL and ASR Externalizing factor in the current study was low (r = .28).

Analysis

All analyses were conducted in SPSS 26 (IBM Corp, 2019).

Correlation analysis. First, Pearson product moment correlation coefficients were computed for the sum scores of all measures (except for the ESI general inhibition, for which factor scores were used). In order to investigate the association between the predictor variables and differences between the informant- and self-report outcome measures (ABCL and ASR scores respectively), Pearson correlations were computed for absolute and raw difference scores between the ABCL and ASR, as well. These difference scores were computed by subtracting ASR Externalizing sum scores from ABCL Externalizing sum scores.

Incremental validity analysis. A multiple regression analysis was conducted twice. Both regression models were controlled for age and gender. In the first analysis the outcome variable was the total score of the ASR, which is the self-report version of the ABCL; in the second analysis the total scores of the informant version of the ABCL was the outcome variable. Predictor variables were ESI general Disinhibition (Ext) factor scores, PPI-R Self-Centered Impulsivity (SI) total scores, and Barrat Impulsivity (BIS) total scores. Ext factor scores were extracted from the database used by Soe-Agnie et al. (2020). Detailed information is provided by Soe-Agnie et al. (2020). In short, the factor scores were computed by Soe-Agnie et al. (2020) through confirmatory factor analysis, using maximum likelihood estimation. These analyses were conducted in R (“R Core Team,” 2018), using the Lavaan package (version 3.5.0; Rosseel, 2012). In order to assess the variability of our sample regression coefficients, we computed the confidence interval of the unstandardized regression coefficients employing a bootstrap procedure with 1,000 iterations (Field, 2013). Bias corrected accelerated confidence intervals were computed (Field, 2013) to produce a more accurate estimation, in case the distribution of the bootstraps was non-normal.

Preliminary analysis mediating role Virtuous Responding. Since the VR scores were used as the mediating variable and the reliability estimate of the VR scale was found to be low in the current sample, the mediated regression should be treated as a preliminary analysis. The mediated regression analysis was conducted twice, using the same predictor (Ext, SI, BIS) and outcome variables (ABCL, ASR) as was used in the multiple regression analyses. We used the Process macro (version 3.5; Hayes, 2020), which is a syntax that modifies the SPSS program in order to allow elaborate mediator analyses. Similar to the aforementioned multiple regression analysis, we conducted a bootstrap analysis using 1,000 bootstrap samples. Direct and indirect effects were computed for the ESI, SI and BIS separately, by testing three mediation models with each individual impulsivity measure as a predictor variable and using the other impulsivity measures and the control variables as covariates. In mediation analysis, the direct effect is the relationship between the predictor and outcome variables, when the mediator variable is held constant. The indirect effect is the relationship between the predictor and outcome variables when the predictor is held constant and the mediator variable is not (e.g. Field, 2013, p. 409; Hayes, 2018, p. 79). We considered a mediation effect to be significant when the confidence intervals of the indirect effect did not include zero (Field, 2013). For the mediated regression analysis, the VR scores were reverse-keyed, meaning that high VR scores represented low levels of self-deception.

Results

Descriptive analyses

Descriptive statistics are presented in Table 2. All mean scores were equivalent between male and female participants. The ABCL Externalizing scores (M_{total sample} = 22.08; SD_{total sample} = 12.37) were significantly higher (t(87) = 2.55, p = .013) than the ASR Externalizing scores (M_{total sample} = 18.58; SD_{total sample} = 7.38). Further inspection of the raw ABCL-ASR difference scores revealed, that the self-report ASR scores were higher than, or equal to, the corresponding informant ABCL scores in 46.6% of the total sample, in 35,7% of the female patients, and in 48.6% of the male patients. This means that these patients reported equal or higher levels of externalizing behaviors in comparison to the staff-members. Overall, VR scale scores were low. The upper boundaries corresponded with T = 45 in the male sample, and with T = 28 in the female sample. These T-scores are based on a Dutch community norm group (Uzieblo et al., 2010).

Table 2. Mean scores and range for total sample and subsamples

	Total (N=93)		Male (n=78)		Female (n=15)
	Mean (SD)	range	Mean (SD)	range	Mean (SD)	range
ESI Ext	227.83 (77.54)	49-393	232.44 (78.82)	49-393	203.53 (70.79)	94-312
SI	85.66 (24.19)	21-152	84.71 (25.41)	21-152	89.53 (17.26)	62-119
VR	16.34 (4.51)	7-29	16.73 (4.48)	8-29	14.67 (4.32)	7-22
BIS	68.36 (11.57)	41-103	67.59 (11.31)	41-89	71.60 (12.68)	57-103
ABCL Externalizing	22.08 (12.37)	0-60	21.85 (12.83)	0-60	22.60 (10.67)	2-41
Intrusive	3.56 (2.95)	0-12	3.53 (3.08)	0-12	3.87 (2.33)	0-7
Rule-breaking	8.68 (5.35)	0-25	8.79 (5.55)	0-25	7.93 (4.42)	1-14
Aggressive	9.78 (5.97)	0-26	9.53 (6.00)	0-26	10.80 (6.00)	1-22
ASR Externalizing	18.58 (7.38)	2-41	18.57 (7.59)	2-41	18.64 (6.39)	11-31
Intrusive	2.77 (1.80)	0-7	2.97 (1.79)	0-7	1.71 (1.49)	0-5
Rule-Breaking	8.41 (3.36)	2-19	8.36 (3.45)	2-19	8.64 (2.93)	5-14
Aggressive	7.40 (3.73)	0-17	7.23 (3.78)	0-17	8.29 (3.41)	4-14
Δ ABCL-ASR	12.07 (7.54)	1-49	12.39 (8.00)	1-49	10.36 (4.20)	4-17

Note. ESI Ext= externalizing Spectrum Inventory general Externalizing. ESI statistics are based on the raw total scores; SI= Psychopathy Personality Inventory Revised Self-Centered Impulsivity scale total scores; BIS=Barrat Impulsiveness Scale-11 total scores; Δ ABCL-ASR are based on absolute differences calculated as ABCL Externalizing total scores minus ASR Externalizing total scores.

Correlation analysis

The correlation matrix is presented in Table 3. The impulsivity scores (SI, ESI and BIS) showed good intercorrelations, following the grading of effect sizes suggested by Evers et al. (2008). These impulsivity scores correlated lower with ABCL scores than with the ASR scores. Analysis of the correlations between (reverse-keyed) VR and the three measures of impulsivity showed adequate correlations. The correlations between (reverse-keyed) VR and absolute and raw differences between ABCL and ASR scores were low.

Table 3. Zero-order Pearson correlations of each measure (n=94)

	ESI	SI	BIS	VR(-)	ABCL	ASR
ESI	-	-	-	-	-	-
SI	.56*	-	-	-	-	-
BIS	.50*	.62*	-	-	-	-
VR(-)	.21*	.44*	.51*	-	-	-
ABCL	.24*	.30*	.21*	.22*	-	-
ASR	.33*	.54*	.52*	.42*	.28*	-
Δ ABCL-ASRabs	.30*	.16	.03	.06	.47	-.04
Δ ABCL-ASRraw	.11	-.14	-.14	-.04	.83	.31

Note. ESI=Externalizing Spectrum Inventory general Disinhibition factor; SI= Psychopathic Personality Inventory-Revised Self-Centered Impulsivity scale; BIS=Barrat Impulsiveness Scale; VR(-)=Psychopathic Personality Inventory-Revised Virtuous Responding scale with high scores representing low levels of virtuous responding; ABCL= Adult Behavior Checklist Externalizing scale, informant version; ASR= Adult Behavior Checklist Externalizing scale, self-report version; Δ ABCL-ASR_abs = ABCL minus ASR absolute difference scores; Δ ABCL-ASR_raw= ABCL minus ASR raw scores; significant levels are two-tailed; *=p<.05.

Predictive validity

Since the bivariate Pearson product-moment correlations revealed high and significant intercorrelations between predictor variables, multicollinearity statistics were checked and considered acceptable, following Miles and Shevlin (2006). The tolerance coefficients were close to one and the variance inflation factor coefficients (VIF) were below two. The results of the regression analysis (Table 4) show that the explained variance of the model containing the three scales representing forms of impulsivity and the control variables gender and age was 43.1% when predicting self-reported (ASR) Externalizing, and 12.9% when predicting the informant-reported (ABCL) Externalizing. Inspection of the standardized beta coefficients revealed that the three impulsivity measures showed differences. The highest standardized betas were found for the SI scale in the models predicting ASR scores, while Ext factor scores showed the highest standardized betas in the model predicting ABCL scores. In the model predicting ABCL scores, all regression coefficients were non-significant, while in the model predicting ASR scores, the regression coefficients of SI and BIS scores were significant.

Table 4. Linear regression model predicting ABCL and ASR Externalizing profile scores from Ext, SI and BIS scores

			ABCL				ASR
Predictor	R^2		b (CI BCa)	β		R^2	b (CI BCa)	β
	.129					.431
Age			-.246 (-.541 - .048)	-.183			-.127 (-.277 - .022)	-.153
Gender			2.953 (-4.239–10.145)	.088			-1.339 (-5.219 – 2.859)	-.067
Ext			2.658 (-.258 – 5.628)	.257			-1.203 (-2.490 - .141)	-.197
SI			.035(-.115 -.186)	.067			.165* (.101 - .224)	.537
BIS			-.063(-.367 -.240)	-.058			.183* (.035 - .354)	.284

Note. ABCL= Adult Behavior Checklist, informant version; ASR= Adult Behavior Checklist, self-report version; * p<.05; p-values are based on 1000 bootstrap samples; CI BCa=confidence interval based on bias corrected accelerated.

Preliminary results mediating role virtuous responding

The results showed no mediation effect of VR on the association between each impulsivity scale and ASR or ABCL externalizing scores, since the confidence intervals of the indirect effects all included zero (not included in Tables).

Discussion

Even though impulsivity has been suggested as an important component in relapse and problem behaviors in the literature, the validity of self-reported impulsivity has scarcely been investigated by using informant-reported behaviors as an outcome measure. Furthermore, only a small number of studies have been published comparing the prediction of self-reported behaviors and informant-reported behaviors. Our initial results showed that a substantial amount of explained variance was lost, when informant-reported behavior was predicted in comparison to the explained variance, when self-reported behavior was predicted. This is consistent with prior research in which effect sizes were lower when informant-reported behavior was used as an outcome variable (Fields et al., 2015), in comparison to the effect sizes when self-reported behaviors were used as outcomes (Cima & Raine, 2009; Hall et al., 2007).

The hypothesis that each impulsivity scale would contribute to the predictive validity equally, was not confirmed. In the model predicting (informant-based) ABCL scores, none of the predictors showed a significant contribution. Although Ext predicted ABCL scores better than it predicted (self-report-based) ASR scores, the contribution was still low and non-significant. In the model predicting (self-report) ASR scores, the ESI showed the lowest and a non-significant contribution. The SI scale of the PPI-R showed the highest, significant contribution, outperforming the BIS even though the latter measure is more widely used and has been validated over the last decades. This result suggests that a heterogeneous measure of impulsivity, such as the PPI-R impulsivity scale which measures impulsivity as well as antisocial tendencies, shows a better predictive validity than a more homogeneous measure of impulsivity, such as the BIS. This result is consistent with studies reporting that heterogeneous assessment instruments have a better predictive validity than homogeneous assessments (Smits et al., 2018).

The current study is a preliminary effort to enhance the knowledge of the predictive power of self-reported impulsivity. Several limitations warrant attention in future research. Firstly, the VR scale showed a low reliability and the mediation analyses showed no significant mediation effect. If the VR scale had shown a mediation effect despite the low reliability, the VR scale could have been considered to be a strong mediator in the investigated regression model. Since this was not the case, one can only conclude that the VR scale was an inappropriate measure of self-deception in the current sample. Marcus et al. (2018) reported that VR showed no moderating effect in the relation between the SI scale and the Triarchic Psychopathy Measure (Patrick, 2010) in an undergraduate sample. In this study, the Cronbach’s alpha was higher (α = .66) than in the current sample, but still insufficient. Taking these findings together, we consider the value of the VR scale as an indicator of self-deception to be highly questionable. In future research, other measures of self-deception may be used, such as the Paulhus Deception Scales (Paulhus,1998).

Secondly, the current results regarding predictive validity are limited to the BIS, SI and Ext measures of impulsivity. It cannot be ruled out that other self-report measures of impulsivity may have stronger predictive power. Bousardt et al. (2016), for instance, reported a significant association between impulsivity as measured by the UPPS (Whiteside & Lynam, 2001; Cyders et al., 2007) and informant-reported inpatient aggression as measured by the SDAS (Bowers et al., 2005).

Thirdly, the intercorrelations between the ASR and ABCL scores were low in comparison to the intercorrelations reported by Achenbach and Rescorla (2003). A possible explanation is that the absolute difference scores between the ABCL and ASR were in the opposite direction in one half of our sample in comparison to the other half. This means that for about half of the patients, self-reported antisocial behavior scores were higher than scores based on informant reports, while for the other half of the patients self-report scores were lower. Apparently, the assumption that patients tend to underreport antisocial behaviors in comparison to staff, as is often described in scientific literature, only applied to half of the current sample.

Implications and future directions

Self-report assessment is the most widely-used formal evaluation method for assessing personality traits in clinical practice, where reliable informants or extensive dossiers are often not available, particularly in The Netherlands. Relevant file information may be available in a Dutch high security forensic setting, but in most other settings, such information may be lacking. This may be especially true for patients in medium and low security forensic settings or addiction care. In these situations, the patient is a convenient and easily accessible source of information. However, the results of the current study did not support the use of the Ext, SI and BIS as predictors of externalizing problem behavior; neither was the use of the VR scale as a bias indicator supported. It is advisable, that more robust outcome measures should be considered in future research. Examples of such outcome measures are incidence reports and recidivism/relapse rates. A prospective design as was applied by Bousardt et al. (2016), which supported the predictive power of self-reported impulsivity, is recommended for future research, as well. Critics of self-report measures usually recommend informant- or interview-based measures or criminal dossier studies as alternatives to self-report assessment (Achenbach, 2006; Hare et al., 1990). Achenbach (2006), for instance, argued that informant versions of self-report measures are to be preferred over validity scales. On the other hand, McGrath et al. (2010) questioned the reliability of informant-based assessment, and recommended using multiple methods (e.g. validity scales, informant-based assessment, physiological measures) before concluding that faking has occurred. In conclusion, the results of the current study indicate that an association between self-report measures of impulsivity and a self-report-based criterion offers insufficient evidence to support predictive validity of these impulsivity measures. The predictive validity of the BIS, SI and Ext as well as the usefulness of the VR scale are questionable. Considering the predictive validity of self-report impulsivity measures, we recommend that future research should use informant-reports, neuroscientific paradigms or recidivism/relapse rates as outcome variables, rather than self-report measures.

Acknowledgments

Supported by Grant 5620076/09/DJI from the Dutch Ministry of Justice and by Tactus Addiction Care. The authors declare that there are no conflicts of interest.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Dutch Custodial Institutions Agency (Dienst Justitiële Inrichtingen), and Tactus Addiction Care (Tactus Verslavingszorg) [5620076/09/DJI].