Examining the Use and Construct Fidelity of Technology-Enhanced Items Employed by K-12 Testing Programs

ABSTRACT

Over the past decade, large-scale testing programs have employed technology-enhanced items (TEI) to improve the fidelity with which an item measures a targeted construct. This paper presents findings from a review of released TEIs employed by large-scale testing programs worldwide. Analyses examine the prevalence with which different types of TEIs are employed and the content areas and grade levels in which they are employed. The analyses apply the Technology-Enhanced Item Utility Framework to examine the fidelity with which current TEIs represent targeted constructs. The analyses indicate that the most common type of TEI employed by testing programs is a drag-and-drop response interaction. Analyses indicate that approximately 40% of the TEIs examined provide a high-level of construct fidelity, while an approximately equal proportion provide low construct fidelity. Finally, the data indicate that a large portion of drag-and-drop items are of low fidelity while other TEI types provide moderate or high fidelity.

Disclosure statemtent

The authors have complied with the American Psychological Association’s ethical standards in the treatment of samples. The authors have no conflict of interest with the work presented in this manuscript.

Notes

¹ Adequate reliability was defined as a minimum of 80 percent agreement between each combination of pairs of raters.

² Measured Progress and American Institute for Research are organizations that develop custom tests for state testing programs and both provide sample items employed by a sub-set of programs they support.