On the Selection of Item Scores or Composite Scores for Clinical Prediction

Abstract

Recent shifts to prioritize prediction, rather than explanation, in psychological science have increased applications of predictive modeling methods. However, composite predictors, such as sum scores, are still commonly used in practice. The motivations behind composite test scores are largely intertwined with reducing the influence of measurement error in answering explanatory questions. But this may be detrimental for predictive aims. The present paper examines the impact of utilizing composite or item-level predictors in linear regression. A mathematical examination of the bias-variance decomposition of prediction error in the presence of measurement error is provided. It is shown that prediction bias, which may be exacerbated by composite scoring, drives prediction error for linear regression. This may be particularly salient when composite scores are comprised of heterogeneous items such as in clinical scales where items correspond to symptoms. With sufficiently large training samples, the increased prediction variance associated with item scores becomes negligible even when composite scores are sufficient. Practical implications of predictor scoring are examined in an empirical example predicting suicidal ideation from various depression scales. Results show that item scores can markedly improve prediction particularly for symptom-based scales. Cross-validation methods can be used to empirically justify predictor scoring decisions.

Keywords:

• Measure scoring
• prediction
• regression
• reliability
• suicide

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was not supported by a grant.

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors would like to thank Dr. William Revelle, two anonymous reviewers, and the Associate Editor Dr. Stephen West for their comments on previous versions of this manuscript. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institution is not intended and should not be inferred.