Machine learning item selection for short scale construction: A proof-of-concept using the SIMS

Abstract

ObjectiveThis proof-of-concept paper provides evidence to support machine learning (ML) as a valid alternative to traditional psychometric techniques in the development of short forms of longer parent psychological tests. ML comprises a variety of feature selection techniques that can be efficiently applied to identify the set of items that best replicates the characteristics of the original test. MethodsIn the present study, we integrated a dataset of 329 participants from published and unpublished datasets used in previous research on the Structured Inventory of Malingered Symptomatology (SIMS) to develop a short version of the scale. The SIMS is a multi-axial self-report questionnaire and a highly efficient psychometric measure of symptom validity, which is frequently applied in forensic settings. Results State-of-the-art ML item selection techniques achieved a 72% reduction in length while capturing 92% of the variance of the original SIMS. The new SIMS short form now consists of 21 items. ConclusionsThe results suggest that the proposed ML-based item selection technique represents a promising alternative to standard psychometric correlation-based methods (i.e. item selection, item response theory), especially when selection techniques (e.g. wrapper) are employed that evaluate global, rather than local, item value.

Keywords:

• Machine learning
• feigned psychopathology
• short scale construction
• SIMS
• psychological test

Availability of data and material

The data that support the findings of this study are available upon request from the first author [G.O.; e-mail [email protected]].

Author contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by G.O. and G.S. The first draft of the manuscript was written by G.O., and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Consent to participate

Informed consent was obtained from all participants included in the study.

Consent for publication

All participants provided informed consent prior to the data collection.

Code availability

Not applicable.

Disclosure statement

No potential conflict of interest was reported by the authors.

Ethical approval

The experimental procedure was approved by the local ethics committee (Board of the Department of Human Neuroscience, Faculty of Medicine and Dentistry, Sapienza University of Rome), in accordance with the Declaration of Helsinki.

Funding

None.