Using Multistage Testing to Enhance Measurement of an English Language Proficiency Test

ABSTRACT

How can one construct a test that provides accurate measurements across the range of performance levels while providing adequate coverage of all of the critical areas of the domain, yet that is not unmanageably long? This paper discusses the approach taken in a linear test of academic English language, and how the transition to a computer-based test allowed for a design that better fit the demands of the test. It also describes the multi-stage adaptive approach that was devised. This approach allows for a test that covers a broad range of performance levels while including items that assess the language of the content areas as described in the English language development standards underpinning the test. The design also allows for a test that is closely tailored to the ability level of the English learner taking the test, and that therefore produces a more precise measure. The efficacy of the design in enhancing measurement of two versions of a high-stakes English language assessments is explored, and the implications of the results are discussed.

Acknowledgments

The authors would like to thank several colleagues for their contributions to this work. Dr. Dorry M. Kenyon provided invaluable guidance in framing this work. This paper benefited from comments by Dr. Mark Chapman, Dr. Alicia Kim, and Dr. H. Gary Cook on an earlier draft. Dr. Jason Kemp helped track down references for the literature review.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ The Kindergarten test is administered in a one-on-one setting, and was not revised along with the other test forms. Therefore, it is not included in the analysis in this paper.

² The Speaking test on linear ACCESS was adaptive, not tiered.

³ Criteria for exiting students from English language services are set by the individual states, with most states setting the exit criteria at or around an Overall PL score of 5.0.

⁴ Specifically, for linear ACCESS the elementary school clusters are 1–2 and 3–5, while for MST ACCESS they are 1, 2–3, and 4–5.