Variations in Rating Scale Functioning in Assessing Speech Act Production in L2 Chinese

References

• Alemi, M., & Tajeddin, Z. (2013). Pragmatic rating of L2 refusal: Criteria of native and nonnative English teachers. TESL Canada Journal, 30(7), 63–81. doi:10.18806/tesl.v30i7.1152
   Google Scholar
• Bardovi-Harlig, K. (2003). Understanding the role of grammar in the acquisition of L2 pragmatics. In A. Martinez-Flor, E. U. Juan, & A. F. Guerra (Eds.), Pragmatic competence and foreign language teaching (pp. 25–44). Castellón de la Plana, Spain: Universitat Jaume I.
   Google Scholar
• Barkaoui, K. (2010). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54–74. doi:10.1080/15434300903464418
   Web of Science ®Google Scholar
• Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: Fundamental measurement in human sciences (2nd ed.). New York, NY: Routledge.
   Google Scholar
• Chen, X., Ye, L., & Zhang, Y. (1995). Refusing in Chinese. In G. Kasper (Ed.), Pragmatics of Chinese as native and target language (pp. 119–163). Manoa: University of Hawaii.
   Google Scholar
• Chen, Y., & Liu, J. (2016). Constructing a scale to assess L2 written speech act performance: WDCT and e-mail tasks. Language Assessment Quarterly, 13(3), 231–250. doi:10.1080/15434303.2016.1213844
   Web of Science ®Google Scholar
• Eckes, T. (2015). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments (2nd ed.). Frankfurt, Germany: Peter Lang.
   Google Scholar
• Engelhard, G. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York, NY: Routledge.
   Google Scholar
• Grabowski, K. (2013). Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 149–171). New York, NY: Palgrave MacMillan.
   Google Scholar
• Harsch, C., & Martin, G. (2013). Comparing holistic and analytic scoring methods: Issues of validity and reliability. Assessment in Education: Principles, Policy & Practice, 20(3), 281–307.
   Google Scholar
• Hong, W. (2011). Refusals in Chinese: How do L1 and L2 differ? Foreign Language Annals, 44(1), 122–136. doi:10.1111/j.1944-9720.2010.01123.x
   Web of Science ®Google Scholar
• House, J. (1996). Developing pragmatic fluency in English as a foreign language. Studies in Second Language Acquisition, 18(2), 225–252. doi:10.1017/S0272263100014893
   Google Scholar
• Iramaneerat, C., Myford, C. M., Yudkowsky, R., & Lowenstein, T. (2009). Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residence. Advances in Health Sciences Education, 14(4), 575–594. doi:10.1007/s10459-008-9141-3
   PubMed Web of Science ®Google Scholar
• Ishihara, N. (2010). Assessing learners’ pragmatic ability in the classroom. In D. H. Tatsuki & N. R. Houck (Eds.), Pragmatics: Teaching speech acts (pp. 209–228). Alexandra, VA: Teachers of English to Speakers of Other Languages.
   Google Scholar
• Knoch, U. (2010). Investigating the effectiveness of individualized feedback to rating behavior – A longitudinal study. Language Testing, 28(2), 179–200. doi:10.1177/0265532210384252
   Web of Science ®Google Scholar
• Li, S. (2012). The effects of input-based practice on pragmatic development of requests in L2 Chinese. Language Learning, 62(2), 403–438. doi:10.1111/j.1467-9922.2011.00629.x
   Web of Science ®Google Scholar
• Li, S. (2014). The effects of different levels of linguistic proficiency on the development of L2 Chinese request production during study abroad. System, 45, 103–116. doi:10.1016/j.system.2014.05.001
   Web of Science ®Google Scholar
• Linacre, J. M. (1999). Investigating rating scale category utility. Journal of Outcome Measurement, 3(2), 103–122.
   PubMedGoogle Scholar
• Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.
   PubMedGoogle Scholar
• Linacre, M. (2013). A user’s guide to FACETS Rasch-Model computer programs (Version 3.71.0). Chicago, IL. Winsteps.com
   Google Scholar
• Liu, J. (2006). Measuring interlanguage pragmatic knowledge of Chinese EFL learners. Frankfurt am Main, Germany: Peter Lang.
   Google Scholar
• Liu, J., & Xie, L. (2014). Examining rater effects in a WDCT pragmatics test. Iranian Journal of Language Testing, 4(1), 50–65.
   Google Scholar
• McNamara, T. (1996). Measuring second language performance. London, UK: Longman.
   Google Scholar
• Purpura, J. (2016). Second and foreign language assessment. The Modern Language Journal, 100(Suppl.1), 190–208. doi:10.1111/modl.12308
   Google Scholar
• Roever, C. (2007). DIF in the assessment of second language pragmatics. Language Assessment Quarterly, 4(2), 165–189. doi:10.1080/15434300701375733
   Google Scholar
• Roever, C. (2013). Testing implicature under operational conditions. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 43–64). New York, NY: Palgrave MacMillan.
   Google Scholar
• Roever, C., Fraser, C., & Elder, C. (2014). Testing ESL sociopragmatics: Development and validation of a web-based test battery. Frankfurt, Germany: Peter Lang.
   Google Scholar
• Smith, E., Wakely, M., De Kruif, R., & Swartz, C. (2003). Optimizing rating scales for self-efficacy (and other) research. Educational and Psychological Measurement, 63(3), 369–391. doi:10.1177/0013164403063003002
   Web of Science ®Google Scholar
• Sydorenko, T., Maynard, C., & Guntly, E. (2014). Rater behavior when judging language learners’ pragmatic appropriateness in extended discourse. TESL Canada Journal, 32(1), 19–41. doi:10.18806/tesl.v32i1.1197
   Google Scholar
• Taguchi, N. (2007). Task difficulty in oral speech act production. Applied Linguistics, 28(1), 113–135. doi:10.1093/applin/aml051
   Web of Science ®Google Scholar
• Taguchi, N. (2011). Rater variation in the assessment of speech acts. Pragmatics, 21(3), 453–471. doi:10.1075/prag.21.3.08tag
   Web of Science ®Google Scholar
• Taguchi, N., Li, S., & Xiao, F. (2013). Production of formulaic expressions in L2 Chinese: A developmental investigation in a study-abroad context. Chinese as a Second Language Research, 2(1), 23–58. doi:10.1515/caslar-2013-0021
   Google Scholar
• Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford/New York, NY: Oxford University Press.
   Google Scholar
• Timpe Laughlin, V. T., Wain, J., & Schmidgall, J. (2015). Defining and operationalizing the construct of pragmatic competence: Review and recommendations. (ETS Research Report No. RR-15-06).
   Google Scholar
• Walters, S. F. (2007). A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. TESOL Quarterly, 24(2), 155–183.
   Google Scholar
• Winke, P. M., & Teng, C. (2010). Using task-based pragmatics tutorials while studying abroad in China. Intercultural Pragmatics, 7(2), 363–399. doi:10.1515/iprg.2010.016
   Web of Science ®Google Scholar
• Wiseman, C. S. (2012). A comparison of the performance of analytic vs. holistic scoring rubrics to assess L2 writing. Iranian Journal of Language Testing, 2(1), 59–92.
   Google Scholar
• Xi, X., & Mollaun, P. (2011). Using raters from India to score a large-scale speaking test. Language Learning, 61(4), 1222–1255. doi:10.1111/lang.2011.61.issue-4
   Web of Science ®Google Scholar
• Yan, X. (2014). An examination of rater performance on a local oral English proficiency test: A mixed-methods approach. Language Testing, 31(4), 501–527. doi:10.1177/0265532214536171
   Web of Science ®Google Scholar
• Youn, S., & Brown, J. D. (2013). Item difficulty and heritage language learner status in pragmatic tests for Korean as a foreign language. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 98–123). Basingstoke, UK: Palgrave Macmillan.
   Google Scholar
• Youn, S. (2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. doi:10.1177/0265532214557113
   Web of Science ®Google Scholar
• Zhang, Y. (1995). Strategies in Chinese requesting. In G. Kasper (Ed.), Pragmatics of Chinese as native and target language (pp.23-68). Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.
   Google Scholar
• Zhang, J., Xie, N., Wang, S., Li, Y., & Zhang, T. (2010). Report on new HSK development. China Examinations, 9, 38–43.
   Google Scholar