An investigation of integrative and independent listening test tasks in a computerised academic English test

ABSTRACT

This research provided a comprehensive evaluation and validation of the listening section of a newly introduced computerised test, Pearson Test of English Academic (PTE Academic). PTE Academic contains 11 item types assessing academic listening skills either alone or in combination with other skills. First, task analysis helped identify skills important for listening comprehension in academic settings. Aspects analysed included the purpose of assessment tasks, skills/constructs assessed, and task stimuli employed in PTE Academic. The findings indicated that modern technologies enabled PTE Academic, a computer-based test, to assess students’ academic listening abilities in real time using the integration of multi-modal sources. The statistical validation consisted of two stages. Exploratory factor analysis was performed first with a sample of over 5000 students who took PTE Academic, to examine the underlying listening constructs as measured by the scores on the 11 item types item on different listening skills; these scores were subjected to Rasch analysis using CONQUEST. Second, the difficulties of the item types were estimated and the effectiveness of these item types was evaluated by calculating the information function by item type. The study has implications for test developers and test users regarding the interpretation of student performance on listening assessments.

KEYWORDS:

• Computerised test
• academic listening
• integrative and independent tasks
• construct validity

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

Faculty Research Grant, Macau University of Science and Technology [grant number FRG-17-010-UIC].

Notes on contributors

Wei Wei

Wei Wei is an assistant professor at Macau University of Science and Technology. He has a PhD degree in Language Testing and TESOL from the University of Leeds, UK. His research interests are in the fields of language assessment and teacher evaluation.

Ying Zheng

Ying Zheng is an associate professor in Faculty of Humanities at the University of Southampton, UK. She obtained her MEd (2005) and PhD (2009) degrees in cognitive studies from Queen's University, Canada, specializing in second language testing and assessment.