Evaluating the Test Validity of the Comprehensive Assessment of Psychopathic Personality Symptom Rating Sale (CAPP SRS)

Abstract

The Comprehensive Assessment of Psychopathic Personality Symptom Rating Scale (CAPP SRS) is a relatively new measure of psychopathic personality disorder (PPD) based on the CAPP concept map of psychopathy. To investigate the CAPP SRS, we identified the most plausible formal test structure for the test using the framework proposed by Slaney and Maraun, identified an appropriate quantitative characterization of that test structure, and then statistically evaluated it based on analysis of CAPP SRS data collected from a multisite sample of 314 adult male correctional offenders and secure hospital patients in Scotland and England. Overall, the CAPP SRS survived falsification when observed test data were compared to expectations based on the unidimensional monotone latent variable or UMLV model of Holland and Rosenbaum. CAPP SRS composite scores calculated consistent with the UMLV model had good measurement precision and good external validity with respect to scores on an established test of PPD. The findings provide provisional support for the test validity of the CAPP SRS and highlight the importance of theory-driven evaluations of test validity.

Acknowledgments

The authors wish to thank Michael D. Maraun and Kathleen L. Slaney for helpful feedback on an earlier version of this manuscript.

Disclosure statement

The authors declare there have been no previous published disseminations (articles, chapters, or books) based on the dataset used in this paper. The first three authors declare they are authors of the CAPP SRS and provide training in the use of the CAPP SRS for which they receive payment, but they do not receive any royalties from sales of the CAPP SRS.

Notes

1 For critical reviews and discussions, see Haslam et al. (2012) and Maraun and Hart (2016).

2 See https://en.wiktionary.org/wiki/unitary. As noted previously, the domains in the CAPP model and CAPP SRS test are descriptive, referring to the basic personality functions disturbed by symptoms; we did not assume that the domains are distinct entities in a conceptual or mathematical sense.

3 The assumption that PPD is unitary does not entail additional assumptions, such as: PPD symptomatology is homogeneous (i.e., all people with PPD have the same symptoms); PPD can be defined extensionally in terms of symptoms (i.e., a specific set can be identified that are individually necessary and jointly sufficient to specify the disorder); symptoms of PPD are specific to the disorder (i.e., never observed in other disorders); or items of various PPD measures are unidimensional in all possible mathematical senses (i.e., in the senses of principal component analysis, linear or non-linear factor analysis, non-metric multidimensional scaling, d-class latent profile analysis, fractal geometry, network analysis, and so forth).

4 This procedure was specified by Holland and Rosenbaum to control both Type I and II errors. On one hand, setting the test-wise p level lower than is conventional (from .05 to .01) in light of the large number of tests conducted reduces the likelihood of identifying significant negative covariances. But on the other hand, setting the percentage of rejections lower than is conventional (i.e., from 5% to 2%) makes it slightly easier to identify overall rejection of associatedness.

5 If a set of test items is UMLV, then any monotone non-decreasing function of those items will be both associated and conditionally associated. As dichotomization is a monotone non-decreasing function, its use does not pose any problems here (Holland & Rosenbaum, 1986).

6 Samejima’s Graded Response Model is strongly related, both conceptually and mathematically, to the Generalized Partial Credit Model. One difference is that the Graded Response Model is designed specifically for use when item responses are ordered in terms of increasing or decreasing difficulty, as is the case for the CAPP SRS and, in addition, the Graded Response Model tends to yield information functions over a wider range of estimates for the latent trait (Naumenko, 2014).

7 We used α here as a lower-bound estimate of the reliability of CAPP SRS items having already confirmed in Step 4 that items were unidimensional in the UMLV sense; we did not use α, nor can it be used, to evaluate unidimensionality.

8 Based on this reliability, the standard error of measurement for unweighted composites was SEm = 10.49, yielding a 95% CI for observed scores of + 20.56.

9 We say provisionally validated because, as noted by Slaney and Maraun (2008), in principle this is the most a test can hope to achieve.

Additional information

Funding

The research described herein was funded by grants from Research and Development Services, Home Office (United Kingdom), the Chief Scientist Office of the Scotland Office (CZH/4/484) and the Research and Development Directorate, Greater Glasgow Primary Care National Health Service Trust. The opinions expressed herein are those of the authors, not official policy statement of the agencies that provided funding or assistance.