Science and Art of Setting Performance Standards and Cutoff Scores in Kinesiology

Abstract

Setting standards and cutoff scores is essential to any measurement and evaluation practice. Two evaluation frameworks, norm-referenced (NR) and criterion-referenced (CR), have often been used for setting standards. Although setting fitness standards based on the NR evaluation is relatively easy as long as a nationally representative sample can be obtained and regularly updated, it has several limitations—namely, time dependency, population dependence, discouraging low-level performers, and favoring advantaged or punishing disadvantaged individuals. Fortunately, these limitations can be significantly eliminated by employing the CR evaluation, which was introduced to kinesiology by Safrit and colleagues in the 1980s and has been successfully applied to some practical problems (e.g., set health-related fitness standards for FITNESSGRAM^®). Yet, the CR evaluation has its own challenges, e.g., selecting an appropriate measure for a criterion behavior, when the expected relationship between the criterion behavior and a predictive measure is not clear, and when standards are not consistent among multiple field measures. Some of these challenges can be addressed by employing the latest statistical methods (e.g., test equating). This article provides a comprehensive review of the science and art of setting standards and cutoff scores in kinesiology. After a brief historical overview of the standard-setting practice in kinesiology is presented, a case analysis of a successful CR evaluation, along with related challenges, is described. Lessons learned from past and current practice as well as how to develop a defendable standard are described. Finally, future research needs and directions are outlined.

Keywords:

• classification
• criterion-referenced
• decision making
• evaluation
• testing

Notes

¹ After the 2012–2013 school year, the NR-based President's Challenge Physical Fitness Test will be replaced by the Presidential Youth Fitness Program (i.e., Fitnessgram)

² Although it may not be the best practice to use an alternative field test to correlate with a health outcome measure, it was done sometimes in practice, as illustrated in Part D of Figure 5.

³ Unfortunately, because BMI is required for the 1-mile run/walk prediction equation and many states or schools do not allow BMI measurements of their students, the standard set for the aerobic capacity based on this advanced method will be dropped from the next version of the HFZ (http://www.cde.ca.gov/ta/tg/pf/healthfitzones.asp).