The Multimodal Listening Test in a High-Stakes Context: Gender-Neutral or not?

References

• Abitreenit. (2018, March 16). The English test of the matriculation examination. Spring 2018, advanced syllabus. http://yle.fi/plus/abitreenit/2018/kevat/EA-fi/EA-fi/index.html
   Google Scholar
• Alavi, S. M., Kaivanpanah, S., & Masjedlou, A. P. (2018). Validity of the listening module of international English Language Testing System: Multiple sources of evidence. Language Testing in Asia, 8(8), 1–17). https://doi.org/https://doi.org/10.1186/s40468-018-0057-4
   Google Scholar
• Anckar, J. (2011). Assessing foreign language listening comprehension by means of the multiple-choice format: Processes and products [Doctoral dissertation, University of Jyväskylä]. Jyväskylä Studies in Humanities. http://urn.fi/URN:ISBN:978-951-39-4410-0
   Google Scholar
• Aryadoust, V., Goh, C. C., & Kim, L. O. (2011). An investigation of differential item functioning in the MELAB listening test. Language Assessment Quarterly, 8(4), 361–385. https://doi.org/https://doi.org/10.1080/15434303.2011.628632
   Web of Science ®Google Scholar
• Aryadoust, V., Ng, L. Y., & Sayama, H. (2020). A comprehensive review of Rasch measurement in language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6–40. https://doi.org/https://doi.org/10.1177/0265532220927487
   Web of Science ®Google Scholar
• Aryadoust, V. (2012). Differential item functioning in while-listening performance tests: The case of the international English Language Testing System (IELTS) listening module. International Journal of Listening, 26(1), 40–60. https://doi.org/https://doi.org/10.1080/10904018.2012.639649
   Google Scholar
• Aryadoust, V. (2017). The listening test of the Internet‐Based Test of English as a Foreign Language (TOEFL iBT). In D. L. Worthington & G. D. Bodie (Eds.), The sourcebook of listening research: Methodology and measures (pp. 592–598). John Wiley & Sons, Inc.
   Google Scholar
• Banerjee, J., & Papageorgiou, S. (2016). What’s in a topic? Exploring the interaction between test-taker age and item content in high-stakes testing. International Journal of Listening, 3(1–2), 8–24. https://doi.org/https://doi.org/10.1080/10904018.2015.1056876
   Google Scholar
• Batty, A. O. (2020). An eye-tracking study of attention to visual cues in L2 listening tests. Language Testing 38(4) , 511–535. https://doi.org/https://doi.org/10.1177/0265532220951504
   Web of Science ®Google Scholar
• Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer Science & Business Media.
   Google Scholar
• Boone, W. J., & Staver, J. R. (2020). Advances in Rasch analyses in the human sciences. Springer.
   Google Scholar
• Fan, J., & Bond, T. (2019). Unidimensionality and local Independence. In V. Aryadoust & M. Rachelle (Eds.), Quantitative data analysis for language assessment (Volume I): Fundamental techniques (pp. 83–102). Routledge.
   Google Scholar
• Fan, J., & Knoch, U. (2019). Fairness in language assessment: What can the Rasch model offer? Papers in Language Testing and Assessment, 8(2), 117–142. http://www.altaanz.org/uploads/5/9/0/8/5908292/8_2_s5_fan_and_knoch.pdf
   Web of Science ®Google Scholar
• Ferne, T., & Rupp, A. A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4(2), 113–148. https://doi.org/https://doi.org/10.1080/15434300701375923
   Google Scholar
• Finnish National Agency for Education. (2015). Lukion opetussuunnitelman perusteet 2015 [National core curriculum for general upper secondary schools]. Finnish National Agency for Education.
   Google Scholar
• Harding, L. (2012). Accent, listening assessment and the potential for a shared-L1 advantage: A DIF perspective. Language Testing, 29(2), 163–180. https://doi.org/https://doi.org/10.1177/0265532211421161
   Web of Science ®Google Scholar
• Härmälä, M., Huhtanen, M., & Puukko, M. (2014). Englannin kielen A-oppimäärän oppimistulokset perusopetuksen päättövaiheessa 2013. [Learning outcomes in advanced syllabus English at the end of basic education 2013]. Finnish National Evaluation Centre. Publications 2014: 2.
   Google Scholar
• Härmälä, M., Huhtanen, M., Puukko, M., & Marjanen, J. (2019). A-Englannin oppimistulokset 7. Luokan alussa 2018. [Learning outcomes in advanced syllabus English at the beginning of grade 7]. Finnish National Evaluation Centre. Publications 13: 2019.
   Google Scholar
• Hilden, R., von Zansen, A., & Laihanen, E. ( accepted 2021). Studioista steissille - Multimodaaliset kuuntelutehtävät ylioppilastutkinnon pitkien oppimäärien kielikokeissa 2018. [Out to the world from recording studio – Multimodal tasks in the Matriculation Examination language tests of advanced syllabi in 2018]. Suomen ainedidaktinen seura [Finnish Research Association for Subject Didactics].
   Google Scholar
• Kline, R. B. (2015). Principles and practice of structural equation modeling (4th ed.). Guilford Publishers.
   Google Scholar
• Kunnan, A. J. (2018). Evaluating language assessments. Routledge.
   Google Scholar
• Kupiainen, S., Marjanen, J., & Ouakrim-Soivio, N. (2018). Ylioppilas valintojen pyörteissä. Lukio-opinnot, ylioppilastutkinto ja korkeakoulujen opiskelijavalinta. [Undergraduates facing a myriad of choices. Upper secondary education, Matriculation examination and university admission]. Suomen ainedidaktinen tutkimusseura. [Finnish Research Association for Subject Didactics]. https://helda.helsinki.fi/bitstream/handle/10138/231687/Ad_tutkimuksia_14_verkkojulkaisu.pdf?sequence=1
   Google Scholar
• Leino, K., Ahonen, A. K., Hienonen, N., Hiltunen, J., Lintuvuori, M., Lähteinen, S., Lämsä, J., Nissinen, K., Nissinen, V., Puhakka, E., Pulkkinen, J., Rautopuro, J., Sirén, M., Vainikainen, M.-P., & Vettenranta, J. (2019). PISA 18 ensituloksia: Suomi parhaiden joukossa. (Opetus- ja kulttuuriministeriön julkaisuja; No. 2019:40). Opetus- ja kulttuuriministeriö. http://urn.fi/URN:ISBN:978-952-263-678-2
   Google Scholar
• Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? https://www.rasch.org/rmt/rmt162f.htm
   Google Scholar
• Linacre, J. M. (2017, July 26). Zara: Your high item reliability and your large item separation tell us that your sample of persons (N=180) is large [Comment on the online forum post Person Item separation]. Rasch Measurement Forum. https://raschforum.boards.net/post/3660/thread
   Google Scholar
• Linacre, J. M. (2021a). A user’s guide to WINSTEPS MINISTEP Rasch-model computer programs. https://www.winsteps.com/a/Winsteps-Manual.pdf
   Google Scholar
• Linacre, J. M. (2021b). WINSTEPS (Version 4.7.1.0) [computer program]. Winsteps.com
   Google Scholar
• McNamara, T., Knoch, U., & Fan, J. (2019). Fairness, Justice & Language Assessment. Oxford University Press.
   Google Scholar
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), The American council on education/Macmillan series on higher education. Educational measurement (pp. 13–103). Macmillan Publishing Co, Inc; American Council on Education.
   Google Scholar
• Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: Implications for the dimensionality of language ability. Studies in Educational Evaluation, 68, 100963. https://doi.org/https://doi.org/10.1016/j.stueduc.2020.100963
   Web of Science ®Google Scholar
• National Certificates of Language Proficiency. (n.d.). Finnish national board of education. Retrieved February 15, 2021, from https://www.oph.fi/en/national-certificates-language-proficiency-yki
   Google Scholar
• Park, G. P. (2008). Differential item functioning on an english listening test across gender. TESOL Quarterly, 42(1), 11–123. https://doi.org/https://doi.org/10.1002/j.1545-7249.2008.tb00212.x
   Web of Science ®Google Scholar
• Raquel, M. (2019). The Rasch measurement approach to differential item functioning (DIF) analysis in language assessment research. In V. Aryadoust & M. Raquel (Eds.), Quantitative data analysis for language assessment (volume I): Fundamental techniques (pp. 103–131). Routledge.
   Google Scholar
• Takala, S., & Kaftandjieva, F. (2000). Test fairness: A DIF analysis of an L2 vocabulary test. Language Testing, 17(3), 323–340. https://doi.org/https://doi.org/10.1177/026553220001700303
   Google Scholar
• Tennant, A., & Pallant, J. F. (2007). DIF matters: A practical approach to test if differential item functioning makes a difference. Rasch Measurement Transactions, 20(4), 1082–1084. https://www.rasch.org/rmt/rmt204d.htm
   Google Scholar
• The Matriculation Examination Board. (2020). Toisen kotimaisen ja vieraiden kielten kokeiden määräykset [Regulations for tests of second national languages and foreign languages]. https://www.ylioppilastutkinto.fi/images/sivuston_tiedostot/Ohjeet/Koekohtaiset/kielikokeet_maaraykset_fi.pdf?v=040320
   Google Scholar
• The Matriculation Examination Board. (n.d.). Website of the matriculation examination board. Retrieved March 27, 2021, from https://www.ylioppilastutkinto.fi/en/
   Google Scholar
• von Zansen, A. (2019). Uudenlaista kuullun ymmärtämistä – Kuvan ja videon merkitys ylioppilastutkinnon kielikokeissa [New approaches to assessing listening – Pictures and video in the language tests of the Finnish Matriculation Examination [Doctoral dissertation], University of Jyväskylä]. JYU Dissertations. http://urn.fi/URN:ISBN:978-951-39-7961-4
   Google Scholar
• Wagner, E., & Ockey, G. (2018). An overview of the use of audio-visual texts on L2 listening. In G. J. Ockey & E. Wagner (Eds.), Assessing L2 listening: Moving towards authenticity (pp. 129–144). John Benjamins Publishing Company.
   Google Scholar
• Wright, B., & Linacre, J. M. (1994). Reasonable mean-square fit values. https://www.rasch.org/rmt/rmt83b.htm
   Google Scholar
• Zhu, X., & Aryadoust, V. (2019). Examining test fairness across gender in a computerized reading test: A comparison between A rasch-based DIF-technique and MIMIC. Papers in Language Testing and Assessment, 8(2), 65–90. http://www.altaanz.org/uploads/5/9/0/8/5908292/8_2_s3_zhu_aryadoust.pdf
   Web of Science ®Google Scholar
• Zhu, X., & Aryadoust, V. (2020). An investigation of mother tongue differential item functioning in a high-stakes computerized academic reading test. Computer Assisted Language Learning. https://doi.org/https://doi.org/10.1080/09588221.2019.1704788
   Google Scholar
• Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4(2), 223–233. https://doi.org/https://doi.org/10.1080/15434300701375832
   Google Scholar