Adapting Levels 1 and 2 of Kirkpatrick’s model of training evaluation to examine the effectiveness of a tertiary-level writing course

ABSTRACT

This study adapts Levels 1 and 2 of Kirkpatrick’s model of training evaluation to evaluate learning outcomes of an English as a second language (ESL) paragraph writing course offered by a major Asian university. The study uses a combination of surveys and writing tests administered at the beginning and end of the course. The survey evaluated changes in students’ perception of their skills, attitude, and knowledge (SAK), and the writing tests measured their writing ability. Rasch measurement was applied to examine the psychometric validity of the instruments. The measured abilities were successively subjected to path modeling to evaluate Levels 1 and 2 of the model. The students reported that the module was enjoyable and useful. In addition, their self-perceived level of skills and knowledge developed across time alongside their writing scores but their attitude remained unchanged. Limitations of Kirkpatrick’s model as well as lack of solid frameworks for evaluating educational effectiveness in applied linguistics are discussed.

KEYWORDS:

• Attitude
• Kirkpatrick’s model
• knowledge
• path model
• Rasch measurement
• skills
• writing

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Funding

This work was supported by the Centre for English Language Communication of the National University of Singapore.

Notes on contributors

Vahid Aryadoust

Vahid Aryadoust, PhD, is assistant professor in the English Language and Literature Academic Group of the National Institute of Education, Singapore. His research areas include assessment of language skills and the application of latent variable models, latent class models, and data mining in assessment. He is also interested in the complex dynamical systems theory, idiographic methods for evaluation, and the emergence and evolution of human language(s).