Assessing QuADEM: preliminary notes on a new method for evaluating online language learning courseware

Abstract

In this article, we set out to assess QuADEM (Quality Assessment of Digital Educational Material), one of the latest methods for evaluating online language learning courseware. What is special about QuADEM is that the evaluation is based on observing the actual usage of the online courseware and that, from a checklist of 12 different components, the evaluator is free to pick and choose one or more. In particular, we focus on the QuADEM evaluation of a module of the digital environment Deutsch-Uni Online (DUO) that aims at preparing B1/B2 students for a study semester in Germany. DUO is meant for self-study supported by an online tutor. For our assessment, we observed two respondents during their activities in the online learning module, using think-aloud protocol, video registration, and keystroke logging, and we conducted semistructured postintervention interviews with them. Zooming in on usability, we found that this QuADEM component lacks assessment criteria regarding feedback and task design, both of which turned out to play an important motivational role in our assessment. While both could be added to the QuADEM usability dimension under the denomination “didactic usability,” we suggest that it might be worth reconsidering QuADEM's pick-and-choose approach.

Keywords:

• quality assessment of digital courseware
• usability
• feedback
• task design
• didactic usability