Investigating the Construct of Topical Knowledge in Second Language Assessment: A Scenario-Based Assessment Approach

References

• Alderson, J. C., & Urquhart, A. H. (1985a). The effect of students’ academic discipline on their performance on ESP reading tests. Language Testing, 2, 192–204. doi:10.1177/026553228500200207
   Google Scholar
• Alderson, J. C., & Urquhart, A. H. (1985b). This test is unfair: I’m not an economist. In P. C. Hauptman, R. Leblanc, & M. B. Wesche (Eds.), Second-language performance testing (pp. 25–43). Ottawa, Canada: University of Ottawa Press.
   Google Scholar
• Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: How researchers in learning and literacy talk about knowledge. Review of Educational Research, 61, 315–343. doi:10.3102/00346543061003315
   Web of Science ®Google Scholar
• Almond, R. G., Mislevy, R. J., Steinberg, L., Yan, D., & Williamson, D. (2015). Bayesian networks in educational assessment. New York, NY: Springer.
   Google Scholar
• Anderson, R. C., Reynolds, R. E., Schallert, D. L., & Goetz, E. T. (1977). Frameworks for comprehending discourse. American Educational Research Journal, 14, 367–381. doi:10.3102/00028312014004367
   Web of Science ®Google Scholar
• Bachman, L. F. (2007). What is the construct? The dialectic of abilities and contexts in defining constructs in language assessment. In J. Fox, M. Wesche, D. Bayliss, L. Cheng, C. E. Turner, & C. Doe (Eds.), Language testing reconsidered (pp. 41–71). Ottawa, Canada: University of Ottawa Press.
   Google Scholar
• Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford, UK: Oxford University Press.
   Google Scholar
• Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford, UK: Oxford University Press.
   Google Scholar
• Bennett, R. E. (2010). Cognitively based assessment of, for, and as learning: A preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspectives, 8, 70–91. doi:10.1080/15366367.2010.508686
   Google Scholar
• Bennett, R. E. (2015). The changing nature of educational assessment. Review of Research in Education, 39(1), 370–407. doi:10.3102/0091732X14554179
   Web of Science ®Google Scholar
• Bennett, R. E., & Gitomer, D. H. (2009). Transforming K-12 assessment: Integrating accountability testing, formative assessment and professional support. In C. Wyatt-Smith & J. Cumming (Eds.), Assessment issues of the 21st Century (pp. 43–61). New York, NY: Springer.
   Google Scholar
• Bond, T. G., & Fox, C. M. (2007). Applying the rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
   Google Scholar
• Brinton, D. M., & Snow, M. S. (2017). The evolving architecture of content-based instruction. In M. A. Snow & D. M. Brinton (Eds.), The content-based classroom: New perspectives on integrating language and content (pp. 2–20). Ann Arbor: Michigan University Press.
   Google Scholar
• Canale, M. (1983). On some dimensions of language proficiency. In J. W. Oller Jr. (Ed.), Issues in language testing research (pp. 333–342). Rowley, MA: Newbury House.
   Google Scholar
• Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. doi:10.1093/applin/1.1.1
   Google Scholar
• Carroll, B., Liu, H., & Oh, S. (2015). Design considerations in the assessment of L2 integrated skills through scenarios. The First TC/ETS forum on teaching, learning, and assessment of english language learners, New York, NY.
   Google Scholar
• Chiang, S. C., & Dunkel, P. (1992). The effect of speech modification, prior knowledge, and listening proficiency on EFL lecture learning. TESOL Quarterly, 26, 345–374. doi:10.2307/3587009
   Web of Science ®Google Scholar
• Chung, T., & Berry, V. (2000). The influence of subject knowledge and second language proficiency on the reading comprehension of scientific and technical discourse. Hong Kong Journal of Applied Linguistics, 5(1), 187–222.
   Google Scholar
• Clapham, C. (1996). The development of IELTS: A study of the effect of background knowledge on reading comprehension. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Clarke, M. A. (1980). The short circuit hypothesis of ESL reading: When language competence interferes with reading performance. Modern Language Journal, 64, 203–209.
   Web of Science ®Google Scholar
• Cohen, G. (1983). The psychology of cognition. New York, NY: Academic Press.
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
   Google Scholar
• Dalton-Puffer, C., Nikula, T., & Smit, U. (Eds.). (2010). Language use and language learning in CLIL classrooms. Amsterdam, the Netherlands: John Benjamins Publishing Company.
   Google Scholar
• de Zarobe, Y. R., & Catalán, R. M. J. (2009). Content and language integrated learning: Evidence from research in Europe. Buffalo, NY: Multilingual Matters.
   Google Scholar
• Deane, P., Sabatini, J., Feng, G., Sparks, J., Song, Y., Fowles, M., … Foley, C. (2015). Key Practices in the English Language Arts (ELA): Linking learning theory, assessment, and instruction. ETS Research Report Series, 2015(2), 1–29. doi:10.1002/ets2.12063
   Google Scholar
• Dochy, F. J. R. C., & Alexander, P. A. (1995). Mapping prior knowledge: A framework for discussion among researchers. European Journal of Psychology of Education, 10, 225–242. doi:10.1007/BF03172918
   Web of Science ®Google Scholar
• Dochy, F. J. R. C., Moerkerke, G., & Martens, R. (1996). Integrating assessment, learning, and instruction: Assessment of domain-specific and domain-transcending prior knowledge and progress. Studies in Educational Evaluation, 22, 309–339. doi:10.1016/0191-491X(96)00018-1
   Google Scholar
• Dochy, F. J. R. C., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69, 145–186. doi:10.3102/00346543069002145
   Web of Science ®Google Scholar
• Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Douglas, D. (2005). Testing languages for specific purposes. In E. Hinkel (Ed.), The handbook of research in second language teaching and learning (pp. 857–868). Mahwah, NJ: Erlbaum.
   Google Scholar
• Emery, H. J. (2014). Developments in LSP testing 30 years on? The case of aviation English. Language Assessment Quarterly, 11(2), 198–215. doi:10.1080/15434303.2014.894516
   Web of Science ®Google Scholar
• Flowerdew, L. (2013). English for academic purposes. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics. Blackwell Publishing Ltd. doi:10.1002/9781405198431.wbeal0376
   Google Scholar
• Glaser, R., & De Corte, E. (1992). Preface to the assessment of prior knowledge as a determinant for future learning. In F. J. R. C. Dochy (Ed.), Assessment of prior knowledge as a determinant for future learning (pp. 1–2). Utrecht, the Netherlands: Uitgeverij Lemma B.V.
   Google Scholar
• Gustilo, L., & Magno, C. (2015). Explaining L2 writing performance through a chain of predictors: A SEM approach. The Southeast Asian Journal of English Language Studies, 21, 115–130. doi:10.17576/3L-2015-2102-09
   Google Scholar
• Hailikari, T., Nevgi, A., & Lindblom-Ylänne, S. (2007). Exploring alternative ways of assessing prior knowledge, its components, and their relation to student achievement: A mathematics based case study. Studies in Educational Evaluation, 33, 320–337. doi:10.1016/j.stueduc.2007.07.007
   Google Scholar
• He, L., & Shi, L. (2012). Topical knowledge and ESL writing. Language Testing, 29, 443–464. doi:10.1177/0265532212436659
   Web of Science ®Google Scholar
• Hidalgo, P., Sato, E., & Suzuki, M. (2015). Pearson TELL (Test of English Language Learning): New technologies for assessing language learners. Retrieved from http://www.pearsontell.com/resources
   Google Scholar
• Hill, Y. Z., & Liu, O. L. (2012). Is there any interaction between background knowledge and language proficiency that affects TOEFL iBT® reading performance?. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Hoekje, B. (2016). “Language,” “communication,” and the longing for the authentic in LSP testing. Language Testing, 33, 289–299. doi:10.1177/0265532215607921
   Web of Science ®Google Scholar
• Hönig, I. (2009). Assessment in CLIL: A case study (Unpublished masters thesis). Vienna, Austria: University of Vienna.
   Google Scholar
• Huang, H.-T. D. (2010). Modeling the relationships among topical knowledge, anxiety, and integrated speaking test performance: A structural equation modeling approach (Unpublished doctoral dissertation). Austin, Texas: The University of Texas at Austin.
   Google Scholar
• Huang, H.-T. D., Hung, S.-T. A., & Plakans, L. (2018). Topical knowledge in L2 speaking assessment: Comparing independent and integrated speaking test tasks. Language Testing., 35(1), 27–49. doi:10.1177/0265532216677106
   Web of Science ®Google Scholar
• Jang, E. E., Wagner, M., & Dunlop, M. (2016, June). Construct validation of multimodal scenario-based language assessment (SBLA) tasks for diagnostic placement purposes. Plenary presented at the 38th LTRC Conference, Palermo, Italy.
   Google Scholar
• Jensen, C., & Hansen, C. (1995). The effect of prior knowledge on EAP listening-test performance. Language Testing, 12, 99–119. doi:10.1177/026553229501200106
   Google Scholar
• Khabbazbashi, N. (2017). Topic and background knowledge effects on performance in speaking assessment. Language Testing, 34, 23–48. doi:10.1177/0265532215595666
   Web of Science ®Google Scholar
• Lambert, W. E., & Tucker, G. R. (1972). The bilingual education of children: The St. Lambert Experiment. Rowley, MA: Newbury House.
   Google Scholar
• Lee, H. K., & Anderson, C. (2007). Validity and topic generality of a writing performance test. Language Testing, 24, 307–330. doi:10.1177/0265532207077200
   Google Scholar
• Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transaction, 7, 328.
   Google Scholar
• Linacre, J. M. (2005). A user’s guide to WINSTEPS. Chicago, IL: Winsteps.com.
   Google Scholar
• Linacre, J. M. (2016). Winsteps® (Version 3.92.0) [Computer Software]. Beaverton, Oregon: Winsteps.com. Retrieved from http://www.winsteps.com/
   Google Scholar
• Lipson, M. (1982). Learning information from text: The role of prior knowledge and reading ability. Journal of Reading Behavior, 14, 243–261. doi:10.1080/10862968209547453
   Web of Science ®Google Scholar
• Liu, O. L., Schedl, M., Mallory, J., & Kong, N. (2009). Does content knowledge affect TOEFL iBT reading performance? A confirmatory approach to differential item functioning. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Long, D. R. (1990). What you don’t know can’t help you: An exploratory study of background knowledge and second language listening comprehension. Studies in Second Language Acquisition, 12, 65–80. doi:10.1017/S0272263100008743
   Google Scholar
• Lumley, T., & O’Sullivan, B. (2005). The effect of test-taker gender, audience, and topic on task performance in tape-mediated assessment of speaking. Language Testing, 22, 415–437. doi:10.1191/0265532205lt303oa
   Google Scholar
• Markham, P., & Latham, M. (1987). The influence of religion-specific background knowledge on the listening comprehension of adults second-language students. Language Learning, 37, 157–170. doi:10.1111/j.1467-1770.1987.tb00563.x
   Web of Science ®Google Scholar
• Marzano, R. J. (2004). Building background knowledge for academic achievement: Research on what works in schools. Alexandria, VA: The Association for Supervision and Curriculum Development (ASCD).
   Google Scholar
• Met, M. (1991). Learning language through content: Learning content through language. Foreign Language Annals, 24, 281–295. doi:10.1111/j.1944-9720.1991.tb00472.x
   Web of Science ®Google Scholar
• Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Research Report 03-16. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (1999). Evidence-centered assessment design. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Morgan, C. (2007). Appropriate language assessment in content and language integrated learning. Language Learning Journal, 33(1), 59–67. doi:10.1080/09571730685200121
   Google Scholar
• O’Reilly, T., Deane, P., & Sabatini, J. (2015). Building and sharing knowledge key practice: What do you know, what don’t you know, what did you learn?. ETS Research Report Series, 2015(2), 1–51. doi:10.1002/ets2.12074
   Google Scholar
• O’Reilly, T., & Sabatini, J. (2013). Reading for understanding: How performance moderators and scenarios impact assessment design. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, D. C.: National Academies Press.
   Google Scholar
• Perin, D. (2011). Facilitating student learning through contextualization. New York, NY: Community College Research Center, Teachers College, Columbia University.
   Google Scholar
• Purpura, J. E. (2004). Assessing grammar. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Purpura, J. E. (2015). Broadening the construct of second and foreign language proficiency through scenario-based language assessment. New York, NY: The First TC/ETS Forum on Teaching, Learning, and Assessment of English Language Learners.
   Google Scholar
• Purpura, J. E. (2016). Second and foreign language assessment. The Modern Language Journal, 100(Supplement 2016), 190–208. doi:10.1111/modl.12308
   Google Scholar
• Purpura, J. E. (2017). Assessing meaning. In E. Shohamy, I. G. Or, & S. May (Eds.), Encyclopedia of language and education, Vol. 7. Language testing and assessment (pp. 33–61). Cham, Switzerland: Springer International Publishing.
   Google Scholar
• Sabatini, J., & O’Reilly, T. (2013). Rationale for a new generation of reading comprehension assessments. In B. Miller, L. Cutting, & P. McCardle (Eds.), Unraveling reading comprehension: Behavioral, neurobiological, and genetic components (pp. 100–111). Baltimore, MD: Brookes Publishing.
   Google Scholar
• Sabatini, J., O’Reilly, T., Halderman, L., & Bruce, K. (2014). Integrating scenario-based and component reading skill measures to understand the reading behavior of struggling readers. Learning Disabilities Research & Practice, 29(1), 36–43. doi:10.1111/ldrp.12028
   Web of Science ®Google Scholar
• Schmidt-Rinehart, B. C. (1994). The effects of topic familiarity on second language listening comprehension. The Modern Language Journal, 78, 179–189. doi:10.1111/modl.1994.78.issue-2
   Web of Science ®Google Scholar
• Shapiro, A. M. (2004). How including prior knowledge as a subject variable may change outcomes of learning research. American Educational Research Journal, 41(1), 159–189. doi:10.3102/00028312041001159
   Web of Science ®Google Scholar
• Snow, A., & Katz, A. (2014). Assessing language and content. In A. J. Kunnan (Ed.), The companion to language assessment. John Wiley & Sons, Inc.. doi:10.1002/9781118411360.wbcla122
   Google Scholar
• Song, Y., Deane, P., Graf, E. A., & van Rijn, P. (2013). Using argumentation learning progressions to support teaching and assessments of English language arts. Educational Testing Services R&D Connections, 22, 1–14.
   Google Scholar
• Tedick, D. J. (1990). ESL writing assessment: Subject-matter knowledge and its impact on performance. English for Specific Purposes, 9, 123–143. doi:10.1016/0889-4906(90)90003-U
   Google Scholar
• Upton, T. A., & Connor, U. (2013). Language for specific purposes: Overview. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics. Blackwell Publishing Ltd. doi:10.1002/9781405198431.wbeal0891
   Google Scholar
• Wiersma, W., & Jurs, S. G. (2009). Research methods in education: An introduction (9th ed.). Boston, MA: Pearson.
   Google Scholar
• Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.
   Google Scholar