Assessing the reliability of clinical scales when the data have both nominal and ordinal features: Proposed guidelines for neuropsychological assessments

Abstract

The purpose of this article is to present, for the first time, a comprehensive methodology for assessing the reliability of a clinical scale that is frequently utilized in neuropsychological research and in biomedical studies, more generally. The dichotomous-ordinal scale is characterized by a single category of “absence” and two or more ordinalized categories of “presence” of a symptom trait, state, or behavior, and it also has special properties that need to be understood in order for its reliability to be appropriately assessed. Using the Brief Psychiatric Rating Scale (BPRS) as a clinical example, we cover the principles of expressing scale reliability in terms of a dichotomy (“absence”-“presence” of a given BPRS symptom); as a trichotomy (“none”; “mild to moderate” symptomatology; and “severe” symptomatology); and as the full 7-category dichotomous-ordinal scale: “none,” “very mild,” “mild,” “moderate,” “moderately severe,” “severe,” and “extremely severe.” Criteria are presented that can be used to evaluate which of these three formats produces the most reliable results. Finally, we address, with a second sample, the important issue of replication, or whether the original reliability findings generalize to other independent populations.