Some insights into the impact of affective information when delivering feedback to students

ABSTRACT

The relation between affect-driven feedback and engagement on a given task has been largely investigated. This relation can be used to make personalised instructional decisions and/or modify the affect content within the feedback. However, although it is generally assumed that providing encouraging feedback to students should help them adopt a state of flow, there are instances where those messages might result counterproductive. In this paper, we present a case study with 48 secondary school students using an Intelligent Tutoring System for arithmetical word problem solving. This system, which makes some common assumptions on how to relate affective state with performance, takes into account subjective (user's affective state) and objective information (previous problem performance) to decide the upcoming difficulty levels and the type of affective feedback to be delivered. Surprisingly, results revealed that feedback was more effective when no emotional content was included, and lead to the conclusion that purely instructional and concise help messages are more important than the emotional reinforcement contained therein. This finding shows that this is still an open issue. Different settings present different constraints generating related compounding factors that affect obtained results. This research confirms that new approaches are required to determine when, how and where affect-driven feedback is needed. Affect-driven feedback, engagement and their mutual relation have been largely investigated. Student's interactions combined with their emotional state can be used to make personalised instructional decisions and/or modify the affect content within the feedback, aiming to entice engagement on the task. However, although it is generally assumed that providing encouraging feedback to the students should help them adopt a state of flow, there are instances where those encouraging messages might result counterproductive. In this paper, we analyze these issues in terms of a case study with 48 secondary school students using an Intelligent Tutoring System for arithmetical word problem solving. This system, which makes some common assumptions on how to relate affective state with performance, takes into account subjective (user's affective state) and objective (previous problem performance) information to decide the difficulty level of the next exercise and the type of affective feedback to be delivered. Surprisingly, findings revealed that feedback was more effective when no emotional content was included in the messages, and lead to the conclusion that purely instructional and concise help messages are more important than the emotional reinforcement contained therein. This finding, which coincides with related work, shows that this is still an open issue. Different settings present different constraints and there are related compounding factors that affect obtained results, such as the message's contents and their target, how to measure the effect of the message on engagement through affective variables considering other issues involved, and to what extent engagement can be manipulated solely in terms of affective feedback. The contribution here is that this research confirms that new approaches are needed to determine when, how and where affect-driven feedback is needed. In particular, based on our previous experience in developing educational recommender systems, we suggest the combination of user-centred design methodologies with data mining methods to yield a more effective feedback.

KEYWORDS:

• Affective computing
• feedback
• engagement
• motivation
• intelligent tutoring systems

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Raul Cabestrero http://orcid.org/0000-0003-3697-1303

Olga C. Santos http://orcid.org/0000-0002-9281-4209

Jesus G. Boticario http://orcid.org/0000-0003-4949-9220

David Arnau http://orcid.org/0000-0002-2849-8248

Miguel Arevalillo-Herráez http://orcid.org/0000-0002-0350-2079

Francesc Ferri http://orcid.org/0000-0002-1543-3568

Additional information

Funding

This work has been partially funded by FEDER and several Spanish projects: MINECO through projects TIN2014-59641-C2-{1,2}-P and UNED through project GID2016-34.