Examining the Scoring of Content Integration in a Listening-Speaking Test: A G-Theory Analysis

ABSTRACT

Communication in the real world often entails the interpretation, evaluation, and integration of content from different sources. However, it appears that the ability to integrate content into discourse has not been explicitly scored for in existing studies. This study operationalizes content integration in the analytic scoring of a listening-speaking test in Chinese. International students who were non-native speakers of Chinese took the test that comprised two retell tasks and an oral presentation linked by an academic scenario. They were scored for content integration, organization, delivery, and language control for all three tasks. Multivariate generalizability theory (G-theory) was used to investigate the functioning of content integration in the analytic rubric of the retell tasks and oral presentation respectively. Overall, this study aimed to illuminate issues on dependability and construct validity for the two task types, focusing on content integration particularly. The findings suggested that content integration functioned differently to some extent when compared with the other dimensions studied.

Acknowledgments

I am indebted to Professor Emeritus James Purpura and Dr. Kirby Grabowski for their guidance on this study. I am also grateful to Professor Yuanman Liu, Dr. Mingming Shao and the research participants for their help. Lastly, I would like to thank the reviewers for their detailed and valuable feedback which helped to improve the quality of this paper.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Funding

This study was supported by Confucius China Studies Program Joint Research Ph.D. Fellowship awarded by Confucius Institute Headquarters (Hanban).