Scores From Riding's Cognitive Styles Analysis Have Poor Test–Retest Reliability

Abstract

Background: The Cognitive Styles Analysis (CSA) purports to assess two cognitive style dimensions, wholist-analytic (WA) and verbalizer-imager (VI). CSA score reliability has not been studied in medical education. Purpose: The objective of this study was to evaluate test–retest reliability and learner-perceived accuracy of CSA scores. Method: CSA scores were measured twice and perceived accuracy of classifications once among 89 family medicine residents, internal medicine residents, and medical students. Results: Mean ± standard deviation interval between tests was 564 ± 136 days. Test–retest correlation for WA scores was 0.30, and for VI scores was 0.12. Upon retesting 44 learners (49%) were classified under a different WA style, and 56 learners (63%) were classified under a different VI style. There were 58 of 73 learners (79%) who agreed or strongly agreed with their WA classification, whereas 51 of 76 (67%) agreed with their VI classification. Conclusions: CSA scores have poor test–retest reliability. Educators may wish to avoid using the CSA and should exercise caution when interpreting CSA scores.

Notes

^a N = 89.

^b Score thresholds are wholist ≤ 1.02, analytic > 1.35; verbalizer ≤ 0.98, imager > 1.09.⁴

^a N = 89.