Scratching Where They Itch: Evaluation of Feedback on a Diagnostic English Grammar Test for Taiwanese University Students

Abstract

Feedback to the test taker is a defining characteristic of diagnostic language testing (Alderson, 2005). This article reports on a study that investigated how much and in what ways students at a Taiwan university perceived the feedback to be useful on an online multiple-choice diagnostic English grammar test, both in general and by students of higher and lower language proficiency. Stage 1 involved questionnaire data from 68 students who rated each item's feedback according to usefulness, and Stage 2 involved interviews with five students as they read the feedback after taking the test. The data from these two stages showed students' overall positive attitude toward the feedback and students' preferences for particular feedback characteristics. The study also found that although higher proficiency test takers found the feedback to be more useful than lower proficiency test-takers, views about the characteristics of good feedback were similar regardless of level. Recommendations for improving diagnostic language test construction and validation are discussed based upon the findings.

ACKNOWLEDGMENTS

The research described in this study was partly funded through the Teaching Excellence funding initiative of the Ministry of Education, Taiwan. The authors would like to express sincere thanks to their colleagues and assistants who have worked on the OEAS test construction project, including Eugene Chen, Yu-sheng Chen, Mailly Cothran, Zoe Lin, Sara Liu, Marilyn Manno, Brendan Ryan, Sonia Tiao, John Waldrop, and Henry Westheim.

Notes

¹The grammar test reported in this paper is part of a larger test battery, including tests of reading and listening (see the Context of the Study section). Research on students' perceptions of feedback on the other tests is currently being conducted and thus is not reported here.

²We express our appreciation to the anonymous reviewer who raised this distinction in comments on an earlier draft of this article.

³Preliminary findings from analysis of usefulness perceptions of the OEAS listening and reading test feedback suggest that for those two skills, in contrast to grammar knowledge, an item's difficulty does seem to have a noticeable relationship with perceived usefulness of corresponding feedback.

⁴Placement test results were used as a common measure of the non-English-major students' proficiency because they had all taken the same placement test (although in different years) and the university's student intake has not changed substantially in terms of overall performance on the placement test (Sims, 2004). However, it is possible that the English proficiency of the upperclassmen may have changed over time, making their proficiency label problematic.

⁵We express our appreciation to the anonymous reviewer who pointed out this limitation in comments on an earlier draft of this article.