Trials and Tribulations of Teaching Null Hypothesis Significance Testing in the Health Sciences

Abstract

Null hypothesis significance testing (NHST) has become the cornerstone of decision-making in clinical and healthcare research. Statistical significance (p < 0.05) is considered the gold standard for inferring that contextual significance exists. However, such practice is controversial since it was never intended for contextual significance to be inferred based on statistical significance. There have been frequent calls for the abandonment of NHST incorporating the bright-line rule of p < 0.05. The call for a statistics reform represents challenges for the teaching of statistics in the Health Sciences. NHST and p-values are central to traditional undergraduate and postgraduate curricula. It is suggested that whatever the future for NHST, it still needs to be taught. It is important that students appreciate the challenges that inferences based on NHST pose. To avoid such challenges in the future, a greater understanding of the underlying statistical principles is needed. Curricula are typically lacking in these principles, whilst they are difficult concepts based on probability and uncertainty. This may have contributed to the controversial practice of inferring contextual significance from statistical significance. A framework for the teaching of NHST and p-values is presented.

Additional information

Notes on contributors

Philip Sedgwick

Philip Sedgwick is reader in medical statistics and medical education at St. George’s, University of London. He has taught statistics for more than 35 years to students in the health care sciences, at both the undergraduate and postgraduate levels. During that time, he has also been the lead statistician in numerous research projects. His particular interest is early-phase clinical trials. In 2009, Sedgwick was commissioned to write weekly for the leading international general medical journal BMJ, which he did in an educational capacity on topics in medical statistics, epidemiology, and research methods. Over six and half years, he produced more than 300 articles that appeared in a series called “Endgames.” Sedgwick won the Best Contributed Presentation Award from the ASA’s Section on Teaching Statistics in the Health Sciences at JSM in 2018 and 2021.