Quantifying Heuristic Bias: Anchoring, Availability, and Representativeness

ABSTRACT

Construct: Authors examined whether a new vignette-based instrument could isolate and quantify heuristic bias. Background: Heuristics are cognitive shortcuts that may introduce bias and contribute to error. There is no standardized instrument available to quantify heuristic bias in clinical decision making, limiting future study of educational interventions designed to improve calibration of medical decisions. This study presents validity data to support a vignette-based instrument quantifying bias due to the anchoring, availability, and representativeness heuristics. Approach: Participants completed questionnaires requiring assignment of probabilities to potential outcomes of medical and nonmedical scenarios. The instrument randomly presented scenarios in one of two versions: Version A, encouraging heuristic bias, and Version B, worded neutrally. The primary outcome was the difference in probability judgments for Version A versus Version B scenario options. Results: Of 167 participants recruited, 139 enrolled. Participants assigned significantly higher mean probability values to Version A scenario options (M = 9.56, SD = 3.75) than Version B (M = 8.98, SD = 3.76), t(1801) = 3.27, p = .001. This result remained significant analyzing medical scenarios alone (Version A, M = 9.41, SD = 3.92; Version B, M = 8.86, SD = 4.09), t(1204) = 2.36, p = .02. Analyzing medical scenarios by heuristic revealed a significant difference between Version A and B for availability (Version A, M = 6.52, SD = 3.32; Version B, M = 5.52, SD = 3.05), t(404) = 3.04, p = .003, and representativeness (Version A, M = 11.45, SD = 3.12; Version B, M = 10.67, SD = 3.71), t(396) = 2.28, p = .02, but not anchoring. Stratifying by training level, students maintained a significant difference between Version A and B medical scenarios (Version A, M = 9.83, SD = 3.75; Version B, M = 9.00, SD = 3.98), t(465) = 2.29, p = .02, but not residents or attendings. Stratifying by heuristic and training level, availability maintained significance for students (Version A, M = 7.28, SD = 3.46; Version B, M = 5.82, SD = 3.22), t(153) = 2.67, p = .008, and residents (Version A, M = 7.19, SD = 3.24; Version B, M = 5.56, SD = 2.72), t(77) = 2.32, p = .02, but not attendings. Conclusions: Authors developed an instrument to isolate and quantify bias produced by the availability and representativeness heuristics, and illustrated the utility of their instrument by demonstrating decreased heuristic bias within medical contexts at higher training levels.

KEYWORDS:

• medical education
• medical decision-making
• heuristics
• cognitive bias
• educational assessment

Acknowledgments

We thank Dr. Nabila Dahodwala, Dr. Rachel Goldmann Gross, and Professor Barry Schwartz for their assistance in the conceptualization and design of this project.