Investigating the Construct of Topical Knowledge in Second Language Assessment: A Scenario-Based Assessment Approach

ABSTRACT

To effectively achieve communicative goals, second language (L2) learners need not only the necessary L2 knowledge, skills, and abilities, but also the relevant topical knowledge. However, in L2 assessment contexts, very few existing tests-in-operation have examined L2 learners’ topical knowledge as part of their language performance. This study investigates the construct of topical knowledge in a scenario-based language assessment (SBLA) that simulates real-life language use of building and sharing knowledge. 41 L2 learners at the intermediate to advanced levels from an adult ESL program participated in the study. Topical knowledge was operationalized as content knowledge and lexical knowledge associated with the theme of the SBLA. Descriptive statistics, correlations, Rasch analysis, and repeated measures ANOVAs were used to examine the construct validity of topical knowledge in the SBLA. The results show that L2 learners’ topical knowledge encompasses both content and lexical knowledge, but they function differently in the process of building and sharing knowledge. Nevertheless, both play a role in the performance outcomes and should be recognized as an integral component of L2 proficiency. The findings also lend evidentiary support for the use of a highly-contextualized SBLA to broaden our understanding of the underlying construct of L2 communicative ability.

Acknowledgments

The study was funded by British Council Assessment Research Award, and I wish to express gratitude to British Council for its generous financial support. Special thanks go to Dr. James Purpura for his ongoing guidance and support and for his valuable feedback on the earlier drafts of this paper. Thanks also go to the anonymous reviewers for their insightful and helpful comments, and to the CEP for their assistance in the participant-recruitment process. Finally, I thank Ashish Banerjee for always being my first reader and letting me know whether my academic storytelling is making sense.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

¹ For more information on SBA, see https://www.ets.org/research/topics/reading_for_ understanding/assessments.

Additional information

Funding

This work was supported by the 2016 British Council Assessment Research Award.