Moving college health research forward: Reconsidering our reliance on statistical significance testing

Abstract

Understanding the unique health needs of college students and establishing best practices to address them depend, heavily, on the inherent quality and contribution of the research identifying these needs. College health-focused publications currently exemplify less than ideal statistical reporting practices. Specifically, college health practitioners and researchers continue to rely heavily upon null hypothesis significance testing (NHST) as the sole standard for effectiveness, validity, and/or replicability of scientific studies, even though NHST itself was not designed for such purposes. Herein we address the following questions: (a) What is NHST? (b) What are the inherent limitations of NHST? (c) What are recommended alternatives to NHST? and (d) How can editorial policies promote adopting NHST alternatives? Using college health data from the CORE 2011 Alcohol and Drug survey, we provide a heuristic example demonstrating how effect sizes do not suffer from the same limitations as NHST.

Keywords:

• College health
• effect size
• null hypothesis significance testing
• significance testing

Notes

1 The critical alpha level for a significant p-value was set at the commonly accepted p <.05.

2 Cohen’s d is an effect size used to determine the standardized difference between two means. In social sciences, Cohen’s d is an appropriate effect size for two-mean comparisons.