Basic Attributes Test (BAT) Retest Performance

Abstract

The Basic Attributes Test (BAT) contributes to a U.S. Air Force pilot selection composite known as the Pilot Candidate Selection Method (PCSM). When PCSM was operationally implemented in 1993, no retests were permitted on the BAT. To determine the effects of retesting on mean score change and reliability, the BAT was administered to 477 college students who were then retested after 2 weeks, 3 months, or 6 months. Several important findings were observed. First, about 70% of the students exhibited score improvement on retest, regardless of length of retest interval. Those who performed poorly on the 1st test generally exhibited larger improvements than those who performed well on the 1st test. Second, practice effects diminished as the length of the retest interval increased. For a 6-month retest interval, it was expected that the mean increase in PCSM scores would be about 6 percentile points. The results suggest that BAT retests could be permitted no less than 6 months after initial testing. Third, and very important, BAT scores demonstrated acceptable reliability. The reliability of the psychomotor composite ranged from .775 to .800, and the reliabilities for the other subtests ranged from .474 to .871.

Notes

1 Preoperational forms of the BAT included several additional tests (Carretta, 1987, 1992a). Some experimental tests were removed during battery development for psychometric reasons (e.g., lack of incremental validity). When the BAT was operationally implemented in 1993, the Mental Rotation test was still administered but was not scored.