Exploring the Validity of a Comprehensive Listening Test to Identify Differences in Primary School Students’ Listening Skills

References

• Acat, M. B., Demiral, H., & Kaya, M. F. (2016). Measuring listening comprehension skills of 5th grade school students with the help of web based systems. International Journal of Instruction, 9(1), 211–224. doi:10.12973/iji.2016.9116a
   Web of Science ®Google Scholar
• Adelmann, K. (2012). The art of listening in an educational perspective: Listening reception in the mother tongue. Education Inquiry, 3(4). 513-534. doi:10.3402/-edui.v3i4.22051
   Google Scholar
• Agirdag, O. (2010). Exploring bilingualism in a monolingual school system: Insights from Turkish and native students from Belgian schools. British Journal of Sociology of Education, 31(3), 307–321. doi:10.1080/01425691003700540
   Web of Science ®Google Scholar
• Akker, E., & Cutler, A. (2003). Prosodic cues to semantic structure in native and nonnative listening. Bilingualism: Language and Cognition, 6(2), 81–96. doi:10.1017/S1366728903001056
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: Author.
   Google Scholar
• Andringa, S., Olsthoorn, N., van Beuningen, C., Schoonen, R., & Hulstijn, J. (2012). Determinants of success in native and non-native listening comprehension: An individual differences approach. Language Learning, 62(s2), 49–78. doi:10.1111/j.1467-9922.2012.00706.x
   Web of Science ®Google Scholar
• Aryadoust, V., Goh, C. C. M., & Kim, L. O. (2011). An investigation of differential item functioning in the MELAB listening test. Language Assessment Quarterly, 8(4), 361–385. doi:10.1080/15434303.2011.628632
   Web of Science ®Google Scholar
• Asia, A., Tolla, A., & Salam, S. (2019). Indonesian vocabulary mastery of early-aged children in Paud Melati Makassar. Journal of Language Teaching and Research, 10(3), 535–540. doi:10.17507/jltr.1003.17
   Google Scholar
• Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford, England: Oxford University Press.
   Google Scholar
• Beall, L. M., Rosier-Gill, J., Tate, J., & Matten, A. (2008). State of the context: Listening education. The International Journal of Listening, 22(2), 123–132. doi:10.1080/10904010802174826
   Google Scholar
• Berninger, V., & Abbott, R. (2010). Listening comprehension, oral expression, reading comprehension and written expression: Related yet unique language systems in grades 1, 3, 5, and 7. Journal of Educational Psychology, 102(3), 635–651. doi:10.1037/a0019319
   PubMed Web of Science ®Google Scholar
• Blommaert, J., & Van Avermaet, P. (2008). Taal, onderwijs en de samenleving: De kloof tussen beleid en realiteit. [Language, education and the society: The gap between policy and reality]. Antwerp, Belgium: EPO.
   Google Scholar
• Bourdeaud'hui, H., Aesaert, K., & van Braak, J. (2020). Identifying student-and class-level correlates of sixth-grade students’ listening comprehension. L1 Educational Studies in Language and Literature, 20, 1–38. doi: 10.17239/L1ESLL-2020.20.01.04
   Google Scholar
• Bourdieu, P. (1991). Language and symbolic power. Cambridge, England: Harvard University Press.
   Google Scholar
• Brown, G. (2008). Selective listening. System, 36(1), 10–21. doi:10.1016/-j.system.2007.11.002
   Web of Science ®Google Scholar
• Brownell, J. (2016). Listening: Attitudes, principles, and skills (6th ed.). New York, NJ: Pearson.
   Google Scholar
• Buck, G. (2001). Assessing listening. Cambridge, England: Cambridge University Press.
   Google Scholar
• Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. doi:10.18637/-jss.v048.i06
   Web of Science ®Google Scholar
• Chang, A. C., & Read, J. (2013). Investigating the effects of multiple-choice listening test items in the oral versus written mode on L2 listeners’ performance and perceptions. System, 41(3), 575–586. doi:10.1016/j.system.2013.06.001
   Web of Science ®Google Scholar
• Chapelle, C. A. (1999). Validity in language assessment. Annual Review of Applied Linguistics, 19, 254–272. doi:10.1017/S0267190599190135
   Google Scholar
• Crocker, L., & Algina, J. (2006). Classical and modern test theory. Boca Raton, FL: H. B. Jovanovich.
   Google Scholar
• Davidson, F. (2004). The identity of language testing. Language Assessment Quarterly: An International Journal, 1(1), 85–88. doi:10.1207/s15434311laq0101_9
   Google Scholar
• de Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: The Guilford Press.
   Google Scholar
• De Champlain, A. F. (2009). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44(1), 109–117. doi:10.1111/j.1365-2923.2009.03425.x
   Web of Science ®Google Scholar
• Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(S1), 5–18. doi:10.1007/s11136-007-9198-0
   PubMed Web of Science ®Google Scholar
• Embretson, S. E., & Reise, S. P. (2013). Item response theory. New York, NY: Psychology Press.
   Google Scholar
• Ferne, T., & Rupp, A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly: An International Journal, 4(2), 113–148. doi:10.1080/15434300701375923
   Google Scholar
• Field, J. (2013). Examining listening. 77–151. Cambridge: Cambridge University Press.
   Google Scholar
• Flowerdew, J., & Miller, L. (2010). Listening in a second language. In A. D. Wolvin (Ed.), Listening and human communication in the 21st century (pp. 158–177). West Sussex, UK: Blackwell.
   Google Scholar
• Ginther, A. (2002). Context and content visuals and performance on listening comprehension stimuli. Language Testing, 2(19), 133–167. doi:10.1191/0265532202lt225oa
   Google Scholar
• Goh, M., & Aryadoust, V. (2016). Learner listening: New insights and directions from empirical studies. International Journal of Listening, 30(1–2), 1–7. doi:10.1080/10904018.2016.1138689
   Google Scholar
• Green, R. (2017). Designing listening tests: A practical approach. London, UK: Palgrave Macmillan.
   Google Scholar
• Hagtvet, B. E. (2003). Listening comprehension and reading comprehension in poor decoders: Evidence for the importance of syntactic and semantic skills as well as phonological kills. Reading and Writing, 16(6), 505–539. doi:10.1023/A:1025521722900
   Google Scholar
• Haladyna, M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–333. doi:10.1207/S15324818AME1503_5
   Web of Science ®Google Scholar
• Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, Calif: Sage Publications, Inc.
   Google Scholar
• Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (2013). Item response theory: Principles and applications. Newbury Park, Calif: Sage Publications, Inc.
   Google Scholar
• Hogan, T. P., Adlof, S. M., & Alonzo, C. N. (2014). On the importance of listening comprehension. International Journal of Speech-language Pathology, 16(3), 199–207. doi:10.3109/17549507.2014.904441
   PubMed Web of Science ®Google Scholar
• Iwankovitsch, R. (2001). The importance of listening. Language Arts Journal of Michigan, 17(2), 5–6. doi:10.9707/2168-149X.1314
   Google Scholar
• Jang, E. E., & Roussos, L. (2009). Integrative analytic approach to detecting and interpreting L2 vocabulary DIF. International Journal of Testing, 9(3), 238–259. doi:10.1080/15305050903107022
   Google Scholar
• Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. doi:10.1111/jedm.12000
   Web of Science ®Google Scholar
• Karimi, M. N., & Naghdivand, R. (2017). Literal and inferential listening comprehension: The role of L1 vs. L2 auditory working memory capacity. Journal of Modern Research in English Language Studies, 4(4), 67–84. doi:10.30479/ELT.2017.1532
   Google Scholar
• Kim, Y.-S. (2015). Direct and mediated effects of language and cognitive skills on comprehension of oral narrative texts (listening comprehension) for children. Journal of Experimental Child Psychology, 141, 101–120. doi:10.1016/j.jecp.2015.08.003
   PubMed Web of Science ®Google Scholar
• Lau, K.-L. (2017). Strategy use, listening problems, and motivation of high- and low-proficiency Chinese listeners. The Journal of Educational Research, 110(5), 503–514. doi:10.1080/00220671.2015.1134421
   Web of Science ®Google Scholar
• Lehto, J. E., & Antilla, M. (2003). Listening comprehension in primary level grades two, four and six. Scandinavian Journal of Educational Research, 47(2), 133–143. doi:10.1080/00313830308615
   Google Scholar
• Lin, S. W., Liu, Y., Chen, S. F., Wang, J. R., & Kao, H. L. (2015). Development of a computer-based measure of listening comprehension of science talk. International Journal of Science and Mathematics Education, 13(6), 1469–1486. doi:10.1007/s10763-014-9559-4
   Web of Science ®Google Scholar
• Lynn, R., & Mikk, J. (2009). Sex differences in reading achievement. Trames, 13(1), 3–13. doi:10.3176/tr.2009.1.01
   Web of Science ®Google Scholar
• Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847–862. doi:10.3758/BRM.42.3.847
   PubMed Web of Science ®Google Scholar
• Marx, A., Heppt, B., & Henschel, S. (2017). Listening comprehension of academic and everyday language in first language and second language students. Applied Psycholinguistics, 38(3), 571–600. doi:10.1017/S0142716416000333
   Web of Science ®Google Scholar
• Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models. Measurement, 11(3), 71–101. doi:10.1080/15366367.2013.831680
   Google Scholar
• McKendry, M. G., & Murphy, V. A. (2011). A comparative study of listening comprehension measures in English as an additional language and native English-speaking primary school children. Evaluation & Research in Education, 24(1), 17. doi:10.1080/0950079.201.531702
   Google Scholar
• Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2). doi:10.1080/5-11.10.3102/2F0013189-X0-18002005
   Google Scholar
• Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256. doi:10.1177/026553229601300302
   Google Scholar
• Neumannn, I., Neumann, K., & Nehm, R. (2011). Evaluating instrument quality in science education: Rasch-based analyses of a nature of science test. International Journal of Science Education, 33(10), 1373–1405. doi:10.1080/09500693.2010.511297
   Web of Science ®Google Scholar
• Ockey, G. J. (2007). Construct implications of including still image or video in computer-based listening tests. Language Testing, 24(4), 517–537. doi:10.1177/026553-2207080771
   Google Scholar
• Oduolowu, E. O., & Oluwakemi, A. E. (2014). Effect of storytelling on listening skills of primary one pupil in Ibadan north local government area of Oyo State, Nigeria. International Journal of Humanities and Social Science, 4(9), 100–107.
   Google Scholar
• Oliveri, M. E., Ercikan, K., & Zumbo, B. D. (2013). Analysis of sources of latent class DIF in international assessments. International Journal of Testing, 13(3), 272–293. doi:10.1080/15305058.2012.738266
   Google Scholar
• Özbay, M. (2010). Turkish education neglected area: Listening training. Turkish Language Teaching Articles (pp. 191–201). Ankara, Turkey: Oncu Book.
   Google Scholar
• Papadopoulou, D., & Clahsen, H. (2003). Parsing strategies in LI and L2 sentence processing: A study of relative clause attachment in Greek. Studies in Second Language Acquisition, 25(4), 501–528. doi:10.1017/S0272263103000214
   Google Scholar
• Piolat, A., Olive, T., & Kellogg, R. T. (2005). Cognitive effort during note taking. Applied Cognitive Psychology, 19(3), 291–312. doi:10.1002/acp.1086
   Web of Science ®Google Scholar
• Potocki, A., Ecalle, J., & Magnan, A. (2012). Narrative comprehension skills in 5-Year-Old children: Correlational analysis and comprehender profiles. The Journal of Educational Research, 106(1), 14–26. doi:10.1080/00220671.2012.667013
   Web of Science ®Google Scholar
• Pulinx, R., & Van Avermaet, P. (2014). Linguistic diversity and education. Dynamic interactions between language education policies and teachers’ beliefs. A qualitative study in secondary schools in Flanders (Belgium). Revue Française de Linguistique Appliquée, 19(2), 9–27. doi:10.3917/rfla.192.0009
   Web of Science ®Google Scholar
• Pulinx, R., Van Avermaet, P., & Agirdag, O. (2017). Silencing linguistic diversity: The extent, the determinants and consequences of the monolingual beliefs of Flemish teachers. International Journal of Bilingual Education and Bilingualism, 20(5), 542–556. doi:10.3917/rfla.192.0009
   Web of Science ®Google Scholar
• Reckase, M. D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21(1), 25–36. doi:10.1177/0146621697211002
   Web of Science ®Google Scholar
• Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26(1), 108–116. doi:10.7334/psicothema2013.260
   PubMed Web of Science ®Google Scholar
• Roever, C., & McNamara, T. (2006). Language testing: The social dimension. International Journal of Applied Linguistics, 16(2), 242–258. doi:10.1111/j.1473-4192.2006.00117.x
   Google Scholar
• Rost, M. (2011). Teaching and researching listening (2nd ed.). Harlow, UK: Pearson Education.
   Google Scholar
• Roussos, L. A., Schnipke, D. L., & Pashley, P. J. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24(3), 293–322. doi:10.3102/10769986024003293
   Web of Science ®Google Scholar
• Rupp, A. A., & Leighton, J. P. (Eds.). (2016). The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications. Oxford, UK: John Wiley & Sons.
   Google Scholar
• Santos, S., Viana, F., Prieto, R. I., Brandao, G., & Cadime, I. (2015). Development of listening comprehension tests with narrative and expository texts for Portuguese students. Spanish Journal of Psychology, 18(e5), 1–7. doi:10.1017/sjp.2015.7
   Google Scholar
• Serraj, S., & Noordin, N. (2013). Relationship among Iranian EFL students’ foreign language anxiety, foreign language listening anxiety and their listening comprehension. English Language Teaching, 6(5), 1–12. doi:10.5539/elt.v6n5p1
   Google Scholar
• Siegel, J. (2013). Second language learners’ perceptions of listening strategy instruction. Innovation in Language Learning and Teaching, 7(1), 1–18. doi:10.1080/-17501229.17502011.17653110
   Google Scholar
• Song, X., Southern, G., & Klinger, D. (2015). DIF investigations across groups of gender and academic background in a large-scale high-stakes language test. Papers in Language Testing and Assessment, 4(1),  97–124.
   Web of Science ®Google Scholar
• Spaan, M. (2007). Evolution of a test item. Language Assessment Quarterly, 4(3), 279–293. doi:10.1080/15434300701462937
   Google Scholar
• Sulaiman, N., Muhammad, A. M., Ganapathy, N. N. D. F., Khairuddin, Z., & Othman, S. (2017). Students’ perceptions on using different listening assessment methods: Audio-only and video media. English Language Teaching, 10(8), 93–99. doi:10.5539/elt.v10n8p93
   Google Scholar
• Tanaka, J. S. (1993). Multifaceted conceptions of fit in structural equation models. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 10–39). Newbury Park, CA: Sage.
   Google Scholar
• Tate, R. (2003). A comparison of selected empirical methods for assessing the structure of responses to test items. Applied Psychological Measurement, 27(3), 159–203. doi:10.1177/0146621603027003001
   Web of Science ®Google Scholar
• Tucker, S. (2007). Using remark statistics for test reliability and item analysis. Retrieved from https://www.umaryland.edu/media/umb/cits/umbtestscoring_testanditemanalysis.pdf
   Google Scholar
• Wagner, E. (2008). Video listening tests: What are they measuring? Language Assessment Quarterly, 5(3), 218–243. doi:10.1080/15434300802213015
   Web of Science ®Google Scholar
• Wagner, E. (2013). Assessing listening. In A. J. Kunan (Ed.), Companion to language assessment (Vol. 1, pp. 47–63). Oxford, England: Wiley-Blackwell.
   Google Scholar
• Weir, C. (2005). Language testing and validation: An evidence based approach. Basingstoke, UK: Palgrave MacMillan.
   Google Scholar
• Wolfgramm, C., Suter, N., & Göksel, E. (2016). Examining the role of concentration, vocabulary and self-concept in listening and reading comprehension. International Journal of Listening, 30(1–2), 25–46. doi:10.1080/10904018.2015.1065746
   Google Scholar
• Wolvin, A. D. (2012). Listening in the general education curriculum. International Journal of Listening, 26(2), 122–128. doi:10.1080/10904018.2012.678201
   Google Scholar
• Yanagawa, K., & Green, A. (2008). To show or not to show: The effects of item stems and answering options on performance on a multiple-choice listening comprehension test. System, 36(1), 107–122. doi:10.1016/j.system.2007.12.003
   Web of Science ®Google Scholar
• Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187–213. doi:10.1111/j.1745-3984.1993.tb00423.x
   Web of Science ®Google Scholar
• Yen, W. M., & Fitzpatrick, R. R. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111–153). Westport, CT: American Council on Education and Praeger Publishers.
   Google Scholar
• Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337–347). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
   Google Scholar
• Zubairi, A. M., & Kassim, N. L. A. (2006). Classical and Rasch analysis of dichotomously scored reading comprehension test items. Malaysian Journal of ELT Res, 2, 1–20.
   Google Scholar
• Zumbo, B. D., & Chan, E. K. (2014). Validity and validation in social, behavioral, and health sciences (Vol. 54). New York: Springer International Publishing.
   Google Scholar