Adapting Levels 1 and 2 of Kirkpatrick’s model of training evaluation to examine the effectiveness of a tertiary-level writing course

References

• Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341–358. doi:10.1111/peps.1997.50.issue-2
   Web of Science ®Google Scholar
• Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2003a). Teaching effectiveness: The relationship between reaction and learning criteria. Educational Psychology, 23, 275–285. doi:10.1080/0144341032000060110
   Google Scholar
• Arthur, W., Jr., Bennett, W. J., Edens, P. S., & Bell, S. T. (2003b). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234–245. doi:10.1037/0021-9010.88.2.234
   PubMed Web of Science ®Google Scholar
• Aryadoust, V. (2014). Understanding the growth of ESL paragraph writing skills and its relationships with linguistic features. Educational Psychology: An International Journal of Experimental Educational Psychology—first online, 1–29. doi:10.1080/01443410.2014.950946
   Google Scholar
• Aryadoust, V., Mehran, P., & Alizadeh, P. (2016). Validating a computer-assisted language learning attitude instrument used in Iranian EFL context: An evidence-based approach. Computer Assisted Language Learning Journal, 29(3), 561–595.
   Web of Science ®Google Scholar
• Atkins, P. W. B., & Baddeley, A. D. (1998). Working memory and distributed vocabulary learning. Applied Psycholinguistics, 19, 537–552. doi:10.1017/S0142716400010353
   Web of Science ®Google Scholar
• Baddeley, A. (2003). Working memory and language: An overview. Journal of Communication Disorders, 36, 189–208. doi:10.1016/S0021-9924(03)00019-4
   PubMed Web of Science ®Google Scholar
• Bae, J., & Lee, Y.-S. (2012). Evaluating the development of children’s writing ability in an EFL context. Language Assessment Quarterly, 9(4), 348–374. doi:10.1080/15434303.2012.721424
   Web of Science ®Google Scholar
• Biber, D., Conrad, S., & Reppen, R. (1998). Corpus linguistics: Investigating language structure and use. Cambridge: Cambridge University Press.
   Google Scholar
• Biber, D., Conrad, S., & Cortes, V. (2004). Lexical bundles in university teaching and textbooks. Applied Linguistics, 25(3), 371–405. doi:10.1093/applin/25.3.371
   Web of Science ®Google Scholar
• Biber, D., Nekrasova, T., & Horn, B. (2011). The effectiveness of feedback for L1-English and L2-writing development: A meta-analysis (TOEFL iBT Research Report No TOEFLiBT-14). Princeton, NJ: ETS.
   Google Scholar
• Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective feedback on ESL student writing. Journal of Second Language Writing, 14(3), 191–205. doi:10.1016/j.jslw.2005.08.001
   Web of Science ®Google Scholar
• Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Bouhnik, D., & Deshen, M. (2014). WhatsApp goes to school: Mobile instant messaging between teachers and students. Journal of Information Technology Education: Research, 13, 217–231.
   Google Scholar
• Brown, H. (2004). Language assessment: Principles and classroom practices. White Plains, NJ: Pearson Education.
   Google Scholar
• Bruton, A. (2010). Another reply to Truscott on error correction: Improved situated designs over statistics. System, 38, 491–498. doi:10.1016/j.system.2010.07.001
   Web of Science ®Google Scholar
• Centra, J. A., & Gaubatz, N. B. (2000). Student perceptions of learning and instructional effectiveness in college courses (Research Report No. 9. The Student Instructional Report II). Princeton, NJ: Educational Testing Service.
   Google Scholar
• Chapelle, C. A., Jamieson, J., & Enright, M. K. (Eds.). (2008). Building a validity argument for the test of English as a foreign language. London: Routledge.
   Google Scholar
• Coxhead, A. (2012). Academic vocabulary, writing and English for academic purposes: Perspectives from second language learners. RELC Journal, 43(1), 137–145. doi:10.1177/0033688212439323
   Google Scholar
• Coxhead, A., & Byrd, P. (2007). Preparing writing teachers to teach the vocabulary and grammar of academic prose. Journal of Second Language Writing, 16, 129–147. doi:10.1016/j.jslw.2007.07.002
   Web of Science ®Google Scholar
• de Oliveira, L. C., & Lan, S.-W. (2014). Writing science in an upper elementary classroom: A genre-based approach to teaching English language learners. Journal of Second Language Writing, 25, 23–39. doi:10.1016/j.jslw.2014.05.001
   Web of Science ®Google Scholar
• Deden, A., & Carter, V. K. (1996). Using technology to enhance students’ skills. In E. Jones (Ed.), Preparing competent college graduates: Setting new and higher expectations for student learning. New directions for higher education, 96 (pp. 81–92). San Francisco, CA: Jossey-Bass.
   Google Scholar
• Devereaux, P. J., & Yusuf, S. (2003). The evolution of the randomized controlled trial and its role in evidence-based decision making. Journal of Internal Medicine, 254(2), 105–113. doi:10.1046/j.1365-2796.2003.01201.x
   PubMed Web of Science ®Google Scholar
• Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17. doi:10.1016/j.asw.2014.03.006
   Web of Science ®Google Scholar
• Duncan, T. E., Duncan, S. C., & Strychker, L. A. (2006). An introduction to latent variable growth curve modeling. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Eckes, T. (2011). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Frankfurt, Germany: Peter Lang.
   Google Scholar
• Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis & J. Burstein (Eds.), Handbook of automated essay evaluation: Current applications and new directions (pp.16–35). New York, NY: Routledge.
   Google Scholar
• Engelhard, G., Jr. (2012). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York, NY: Routledge.
   Google Scholar
• Ferris, D. R. (2004). The “Grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime?). Journal of Second Language Writing, 13, 49–62. doi:10.1016/j.jslw.2004.04.005
   Web of Science ®Google Scholar
• Hinkel, E. (2003). Simplicity without elegance: Features of sentences in L1 and L2 academic texts. TESOL Quarterly, 37, 275–301. doi:10.2307/3588505
   Web of Science ®Google Scholar
• Hoang, G., & Kunnan, A. J. (in press). Automated writing instructional tool for English language learners: A case study of MyAccess. Language Assessment Quarterly, 1–18.
   Google Scholar
• Jones, J. (2010). The role of assessment for learning in the management of primary to secondary transition: Implications for language teachers. Language Learning Journal, 38(2), 175–191. doi:10.1080/09571730902928052
   Google Scholar
• Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training and Development, 13, 3–9.
   Google Scholar
• Kirkpatrick, D. L. (1996). Invited reaction: Reaction to Holton article. Human Resource Development Quarterly, 7, 23–25. doi:10.1002/(ISSN)1532-1096
   Google Scholar
• Knoch, U., Rouhshad, A., Oon, S. P., & Storch, N. (2015). What happens to ESL students’ writing after three years of study at an English medium university? Journal of Second Language Writing, 28, 39–52. doi:10.1016/j.jslw.2015.02.005
   Web of Science ®Google Scholar
• Kroll, B., & Reid, J. (1994). Guidelines for designing writing prompts: Clarifications, caveats, and cautions. Journal of Second Language Writing, 3, 231–255.
   Google Scholar
• Lan, Y.-J., Sung, Y.-T., Cheng, C.-C., & Chang, K.-E. (2015). Computer-supported cooperative prewriting for enhancing young EFL learners’ writing performance. Language Learning & Technology, 19(2), 134–155.
   Web of Science ®Google Scholar
• Laufer, B. (2005). Instructed second language vocabulary learning: The fault in the ‘default hypothesis’. In A. Housen & M. Pierrard (Eds.), Investigations in instructed second language acquisition (pp. 311–329). Berlin: Mouton de Gruyter.
   Google Scholar
• Lavolette, E., Polio, C., & Kahng, J. (2015). The accuracy of computer-assisted feedback and students’ responses to it. Language Learning & Technology, 19(2), 50–68.
   Web of Science ®Google Scholar
• Le Grange, L., & Reddy, C. (1998). Continuous assessment: An introduction and guidelines to implementation. Cape Town, South Africa: Juta.
   Google Scholar
• Lee, I., & Coniam, D. (2013). Introducing assessment for learning for EFL writing in an assessment of learning examination-driven system in Hong Kong. Journal of Second Language Writing, 22, 34–50. doi:10.1016/j.jslw.2012.11.003
   Web of Science ®Google Scholar
• Lee, Y. (2013). Collaborative concept mapping as a pre-writing strategy for L2 learning: A Korean application. International Journal of Information and Education Technology, 3(2), 254–258. doi:10.7763/IJIET.2013.V3.275
   Google Scholar
• Linacre, J. M. (1994). Many-facet rasch measurement (2nd ed.). Chicago, IL: MESA Press.
   Google Scholar
• Linacre, J. M. (2013a). A user’s guide to FACETS Rasch-model computer programs. Chicago, IL: Winsteps.com.
   Google Scholar
• Linacre, J. M. (2013b). Facets [Rasch measurement computer program]. Chicago, IL: Winsteps.com.
   Google Scholar
• Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66–81. doi:10.1016/j.jslw.2015.08.011
   Web of Science ®Google Scholar
• Liu, S., & Kunnan, A. (2016). Investigating the application of automated writing evaluation to Chinese undergraduate English majors: a case Study of WriteToLearn. CALICO Journal, 33(1), 71–91.
   Web of Science ®Google Scholar
• Lynch, T., & Anderson, K. (2013). Grammar for academic writing. Edinburgh: English Language Teaching Centre, University of Edinburgh.
   Google Scholar
• Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education.
   Google Scholar
• Pajares, F., & Valiante, G. (1997). Influence of writing self-efficacy beliefs on the writing performance of upper elementary students. Journal of Educational Research, 90, 353–360. doi:10.1080/00220671.1997.10544593
   Web of Science ®Google Scholar
• Polio, C., Fleck, C., & Leder, N. (1998). “If I only had more time:” ESL learners’ changes in linguistic accuracy on essay revisions. Journal of Second Language Writing, 7, 43–68. doi:10.1016/S1060-3743(98)90005-4
   Google Scholar
• Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educational Assessment, Evaluation, and Accountability, 22, 215–225. doi:10.1007/s11092-010-9098-7
   Web of Science ®Google Scholar
• Raimes, A. (2004). Grammar troublespots: A guide for student writers. Cambridge: Cambridge University Press.
   Google Scholar
• Romova, Z. S., & Andrew, M. B. (2011). Teaching and assessing academic writing via the portfolio: Benefits for learners of English as an additional language. Assessing Writing, 16(2), 111–122.
   Google Scholar
• Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104(8), 1525–1567. doi:10.1111/tcre.2002.104.issue-8
   Web of Science ®Google Scholar
• Schoonen, R., Gelderen, A., Stoel, R. D., Hulstijn, J., & Glopper, K. (2011). Modeling the development of L1 and EFL writing proficiency of secondary school students. Language Learning, 61(1), 31–79. doi:10.1111/j.1467-9922.2010.00590.x
   Web of Science ®Google Scholar
• Schumacker, R. E., & Lomax, R. G. (2010). A beginners’ guide to structural equation modeling. New York, NY: Routledge.
   Google Scholar
• Seidel, T., & Shavelson, R. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77(4), 454–499. doi:10.3102/0034654307310317
   Web of Science ®Google Scholar
• Shintani, N., & Ellis, R. (2013). The comparative effect of direct written corrective feedback and metalinguistic explanation on learners’ explicit and implicit knowledge of the English indefinite article. Journal of Second Language Writing, 22(3), 286–306. doi:10.1016/j.jslw.2013.03.011
   Web of Science ®Google Scholar
• Smith, R. M. (1996). A comparison of methods for determining dimensionality in Rasch measurement. Structural Equation Modeling: A Multidisciplinary Journal, 3, 25–40. doi:10.1080/10705519609540027
   Web of Science ®Google Scholar
• Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103–118. doi:10.1016/j.jslw.2009.02.003
   Web of Science ®Google Scholar
• Storch, N., & Tapper, J. (2000). Discipline specific academic writing: What content teachers comment on. Higher Education Research and Development, 19, 337–356. doi:10.1080/758484345
   Google Scholar
• Storch, N., & Tapper, J. (2009). The impact of an EAP course on postgraduate writing. Journal of English for Academic Purposes, 8(3), 207–223. doi:10.1016/j.jeap.2009.03.001
   Google Scholar
• Strobl, C. (2015). Affordances of web 2.0 technologies for collaborative advanced writing in a foreign language. The Computer Assisted Language Instruction Consortium (CALICO), 31(1), 1–18.
   Google Scholar
• Tanaka, J. S. (1987). “How big is big enough?”: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134–146. doi:10.2307/1130296
   Web of Science ®Google Scholar
• Trenkov, L. (2014). Managing teacher-student interaction via WhatsApp platform. Proceedings of EDULEARN14, Barcelona, pp. 6596–6600.
   Google Scholar
• Van Beuningen, C. (2010). Corrective feedback in L2 writing: Theoretical perspectives, empirical insights, and future directions. International Journal of English Studies, 10, 1–27.
   Google Scholar
• Van Buren, M. E., & Erskine, W. (2002). The 2002 ASTD state of the industry report. Alexandria, VA: American Society of Training and Development.
   Google Scholar
• Wexley, K. N., & Latham, G. P. (2002). Developing and training human resources in organizations (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
   Google Scholar
• Xu, C. (2009). Overgeneralization from a narrow focus: A response to Ellis et al. (2008) and Bitchener (2008). Journal of Second Language Writing, 18, 270–275. doi:10.1016/j.jslw.2009.05.005
   Web of Science ®Google Scholar
• Zhang, Z., Yan, X., & Liu, X. (2015). The development of EFL writing instruction and research in China: An update from the international conference on English language teaching. Journal of Second Language Writing, 30, 14–18.
   Web of Science ®Google Scholar