A Theory-Based Comparison of the Reliabilities of Fixed-Length and Trials-to-Criterion Scoring of Physical Education Skills Tests

Abstract

The reliabilities of two types of measurement plans were compared across six hypothetical distributions of true scores or abilities. The measurement plans consisted of a fixed-length plan (FL), where the number of trials for all examinees is fixed in advance, and the trials-to-criterion plan (TTC), where the number of successful trials is fixed, and examinees continue until this criterion is reached. The comparisons revealed that for most hypothetical distributions considered, the FL plan produced higher test reliabilities. In certain cases of negative skewness, however, the TTC plan was superior. Two formulae were presented for the estimation of the reliability of a TTC test.

Key Words:

• reliability
• sequential testing
• trials-to-criterion
• true score distributions