Self-determined motivation for data-based decision-making: A relevance intervention in teacher training

Abstract

While teachers’ core responsibility is to provide high-quality instruction, they are also expected to engage in data-based decision-making (DBDM), e.g., to analyse and use data to improve instruction. We developed a relevance intervention to promote student teachers' self-determined motivation and application intentions for DBDM and implemented it into a large compulsory university course. In a randomized controlled trial, participating students were either repeatedly prompted to reflect about the relevance of DBDM contents (relevance-condition) or asked to summarize DBDM contents (summary-condition). Students in the relevance-condition reported more self-determined forms of motivation, more autonomy-satisfaction, were more willing and self-confident to apply DBDM as teachers than students in the summary-condition. The intervention’s effect on application intentions was fully mediated by an increase in self-determined motivation. Students’ knowledge of DBDM could not be increased by the intervention. Implications for improving university educational training for student teachers are discussed.

Keywords:

• Data-Based decision-making
• self-determination theory
• relevance intervention
• teacher training
• application intentions
• field experiment

PUBLIC INTEREST STATEMENT

Data-based decision-making (DBDM) for teachers means that teachers use systematically collected information for controlling pedagogical processes when making important pedagogical decision. Acquiring the skills required for DBDM does not seem appealing to every student but is nonetheless important. To strengthen a motivation that originates within the learner, we asked teaching students in a regular university course about DBDM several times during the course to select a content taught in the course and reflect about how and why it might be relevant for their future teaching. Compared to a control group, students who had been encouraged to reflect on the content’s relevance were more convinced that DBDM was relevant to their future profession, they described their learning motivation as more intrinsic and autonomous, and reported a stronger willingness to use and apply their skills for DBDM later as a teacher.

University students who aspire to become classroom teachers are often highly motivated to engage with the content of their teaching subjects and to learn how to act pedagogically, to effectively support student learning (e.g., Rutsch et al., 2020). However, student teachers are typically less motivated to engage with learning content that is the basis for data-based decision-making (DBDM, Heitink et al., 2016; Murtonen et al., 2008). DBDM means the systematic gathering, analysis, and evaluation of empirical data, with the goal of examining one’s own pedagogical practice, as well as the improvement of instruction and school effectiveness (Mandinach, 2012). DBDM has become an essential part of the teacher education curriculum to the extent that evidence-based school practice and teacher accountability have been called for by educational policy (e.g., Gogolin et al., 2020, for Germany where our study was conducted; Mandinach & Gummer, 2016, for the U.S.; Schildkamp et al., 2014, for Europe).

DBDM requires knowledge of empirical research methods and statistics, learning content that student teachers often experience as particularly difficult (Murtonen et al., 2008). Later in school, DBDM requires activities which are often not considered by teachers as part of their duties and require an extra time commitment, in addition to the core business of teaching, like for instance, the evaluation of their own teaching or the effectiveness of pedagogical interventions. For these reasons, students’ motivation to engage in DBDM is often dominated by unfavourable, extrinsic, and non-self-directed forms, and their attitudes or intentions to incorporate DBDM into their own practice are often negative, too (Kippers et al., 2018; Murtonen et al., 2008; Sizemore & Lewandowski, 2009). With the goal of promoting more favourable forms of motivation and attitudes toward DBDM, we developed an intervention for students enrolled in a master program in teaching and implemented it into a course on DBDM, which is a compulsory part of the regular university curriculum for student teachers in Germany.

1. Promoting self-determined forms of motivation and positive attitudes towards DBDM

According to Self-Determination Theory (SDT, Deci & Ryan, 2000), students’ motivation can vary in both, strength and quality. The theory postulates that human motivation varies regarding the degree of self-determination, with six forms of motivation being distinguished, depending on what is regulating the person’s behaviour, or in other words, why a person is doing the behaviour. The varying types of behavioural regulations can be ordered on a continuum of self-determination, with a feeling of a complete lack of self-determination on the one hand, and a feeling of full self-determination on the other. The prototype of self-determined motivation is when the activity itself is experienced as rewarding. In our example this would be a student who learns about DBDM just because it satisfies their curiosity and the learning itself gives them pleasure (intrinsic regulation). The extreme on the lower end of the self-determination continuum, amotivation, can be illustrated by a student who perceives no intention and no reason to learn about DBDM at all. In between these two extreme forms of motivation lie four forms of extrinsic motivation (see Figure 1). Here, behaviour is driven by some outside influence or goal. For the two least self-determined forms of extrinsic motivation, external regulation and introjected regulation, outside forces need to be present for the behaviour to be executed, as is illustrated by a student who learns about DBDM mainly because they are afraid not to pass the final test (external regulation) or by a student who feels that it is what is expected from them by others, without truly identifying with the value of DBDM (introjected regulation). The two forms of relatively self-determined forms of extrinsic motivation, identified regulation and integrated regulation, refer to behaviours that originate more strongly within the person. For instance, a student may experience learning about DBDM as not inherently rewarding but still feel autonomously motivated as they identify with its personal importance, i.e., that DBDM can help improve their future teaching (identified regulation). When brought into harmony with other personal values, e.g., an idea of how they want to work as a future teacher, the behaviour may even be perceived as an integrated expression of their identity and core values (integrated regulation). Hence, identified and integrated regulation together are considered as self-determined forms of motivation, despite their partly extrinsic qualities.

Figure 1. The self-determination continuumNote. (Adapted from: Vansteenkiste et al., 2018)

Self-determined motivation has been shown to be related to deep-learning strategies (Bailey & Phillips, 2016; Orsini et al., 2018), increased emotional engagement (Van der Kaap-Deeder et al., 2016), more self-regulated learning (León et al., 2015), and thereby also better academic performance (Bailey & Phillips, 2016; Cerasoli et al., 2014; Orsini et al., 2018; Ratelle et al., 2007). Self-determined forms of motivation have also been shown to be positively related to higher intentions to make use of newly acquired skills outside of the context they were learned in (Chan et al., 2015; Hagger & Chatzisarantis, 2016) and, more specifically, to correlate positively with teachers’ willingness to implement innovations (Gorozidis & Papaioannou, 2014). In contrast, less self-determined forms of motivation were found to be associated with low perseverance, low effort expenditure, weaker performance (Bailey & Phillips, 2016; Cerasoli et al., 2014), and weaker intentions for application (Hagger & Chatzisarantis, 2016; Rump et al., 2017).

2. Strengthening the personal relevance of DBDM

As self-determined forms of motivation have been found to be related to more adaptive student outcomes, it is concerning that students show little interest in the topic of DBDM (Murtonen, 2005; Murtonen et al., 2008) and do not recognize its value for their later professional activity as a teacher (Haberfellner, 2017). According to SDT, a crucial precondition for internalization, i.e., for students to internalize a certain behaviour thus moving towards more self-determined forms of motivation, is that the behaviour possesses relevance for an intrinsic personal goal (Vansteenkiste et al., 2018). After identification with the personal value of a task for an intrinsic future goal, students adapt more self-determined forms of motivation (Deci et al., 1994; Reeve et al., 2002) and effort expenditure happens more autonomously, compared to effort expenditure driven by external pressure (Ryan & Deci, 2017; Vansteenkiste et al., 2018). Provision of a rationale, a statement that conveys information about the meaning and relevance of a learning content for one’s personal life or intrinsic goals, is therefore a typical component of autonomy-supportive teaching practices aiming to support internalization (Ryan & Deci, 2017).

Since DBDM learning content is often experienced as difficult by students (Murtonen et al., 2008), we did not expect that providing a rationale for DBDM would necessarily promote the experience of learning out of pleasure (intrinsic motivation). However, the two extrinsic forms of self-determined motivation, namely identified and integrated regulation, should be promoted by reflecting on the relevance of the learning content: Students will mentally work out the relevance of DBDM for their later professional activity as a teacher, thus recognizing a personal value in it (identified regulation), and integrate this value with other values that make up their identity (integrated regulation; Reeve, 2016; Reeve et al., 2002). The strengthening of these two types of internalized extrinsic motivation might then lead students to develop intentions to apply DBDM, although engagement with the learning content is not driven by experiencing pleasure.

3. Relevance interventions

Interventions where a rationale is provided have already been successfully applied within the expectancy-value framework of motivation (Harackiewicz & Priniski, 2018; Wigfield et al., 2000). Here, making information about the usefulness of course content for students’ personal lives available was shown to strengthen utility-values, thus increasing students’ motivation to engage in the course content, as measured by time or effort invested (e.g., Canning et al., 2017). Expectancy-value theory assumes that utility-values increase motivation as they are instrumental for the attainment of a future goal, irrespective of whether the content of this goal is extrinsic (e.g., helps me pass the exam) or intrinsic (e.g., helps me support students’ learning in my class) and irrespective of whether the rationale is externally provided or self-generated. In contrast, in the SDT framework strengthening motivation is not understood as an increase in quantity, but as a shift to qualitatively different, more self-determined forms. Studies within the SDT-framework have provided evidence that only utility-values relating to an intrinsic, personally important goal promote self-determined forms of motivation (Vansteenkiste et al., 2004, 2018) and that self-determined forms of motivation in turn ensure that learning contents are used outside the context in which they were acquired (Chan et al., 2015; Hagger & Chatzisarantis, 2016). From an SDT perspective, it should also matter whether students can satisfy their basic psychological needs for autonomy, competence, and relatedness, with the satisfaction of the need for autonomy being especially critical for a person to “take ownership” (Ryan & Deci, 2017). Generating rationales oneself should be more autonomy-supportive than providing reasons for a behaviour by an external agent (Steingut et al., 2017). We therefore designed the intervention in such a way that students, while they could freely choose a course content recently covered within the course, were led (a) to reflect about a personally significant goal, namely their professional teaching job, and (b) to self-generate connections to their future professional activity.

Going beyond research conducted within the expectancy-value framework showing effects of utility-value interventions on the strength of motivation, we predicted our intervention to foster self-determined qualities of motivation. As students freely chose the target content and come up themselves with arguments for the relevance of an intrinsic goal, SDT predicts that these motivational effects go along with an increased satisfaction of the need for autonomy. Regarding student’s performance in the course, we expected those who received the intervention to learn more, compared to students from a control group. We further predicted that our intervention would impact students’ intentions to apply course contents as a future teacher. More specifically, we predicted that the two internalized forms of extrinsic motivation would mediate the effect. To the extent that reflecting on the relevance of DBDM strengthens identification with (identified regulation) and integration of reasons (integrated regulation), students’ application intentions should increase, too.

As earlier studies were unclear about a possible effect of rationale provision on students’ self-efficacy we decided to include it in our analyses with no directed hypothesis (e.g., Steingut et al., 2017). We further explored the two other basic psychological needs specified in SDT, competence and relatedness (Ryan & Deci, 2017), checking for possible side effects of our intervention.

4. Research hypotheses

We expected that prompting students to reflect on the relevance of DBDM course contents for an intrinsic personal goal will

• increase the perceived relevance of the course contents (hypothesis 1),

• foster self-determined forms of motivation (hypothesis 2),

• strengthen the satisfaction of autonomy in the course (hypothesis 3),

• increase performance in a knowledge test on the course contents (hypothesis 4),

• strengthen intentions to apply course contents in one’s future job as a teacher (hypothesis 5), mediated by internalized extrinsic motivation (identified and integrated regulation; hypothesis 6)

5. Method

5.1. Procedure, experimental treatment, and participants

5.1.1. Procedure

We conducted a randomized controlled experiment embedded in the regular curriculum of a university course about DBDM for students enrolled in a large university’s master program in teaching. The experimental manipulation was provided via the instruction students received at the beginning of the learning tests. At the beginning of the course, all enrolled students were allocated to either the relevance intervention (relevance-condition) or to a control group (summary-condition) by simple randomization on class level. The course lasted for 12 weeks in total and consisted of four course units (topics: probabilities, diagnostics, intervention and evaluation) which lasted for three weeks each. Each course unit consisted of two lectures and a practical session that were all held by the same lecturer for all students. At the end of each course unit students were offered an online knowledge test, which consisted of multiple-choice questions that covered all the content that had been taught until that time in the course. The first three knowledge tests were voluntary (but participation was recommended). The fourth knowledge test, covering the contents of all course units, was the only obligatory course component and took place at the university. Students needed to answer 50% of the multiple-choice questions correctly to pass the exam. Students could lower their pass mark to 40% when they showed an increase in absolute answers correct from one knowledge test to the next one. This served as an incentive for students to participate in all knowledge tests. At the end of the term, a link to our questionnaire was distributed via email. The questionnaire assessed demographical (age, gender, study program, and track) and psychological variables (perceived relevance of course contents, satisfaction of basic psychological needs, self-determined motivation to learn, self-efficacy for application, application intentions). Students were told that their answers were recorded anonymously and would not affect the evaluation of their coursework, that participation in the survey was voluntary, that they could discontinue the survey at any time, and that neither non-participation nor discontinuation would have negative consequences for them. Participants consented at the beginning of the survey to the scientific use of their data, including their scores on the knowledge tests.

Data collection lasted four weeks with three reminders sent after one, two, and three weeks of the initial invitation, respectively. Participants received a full debrief including preliminary results two months after data collection was finished.

5.1.2. Experimental treatment

Our goal was to develop an intervention that could be integrated into the regular teacher education curriculum. Therefore, reading the rationale and answering the question should take as little time as possible. At the beginning of each knowledge test, students received the following instructions (the topic of the course unit was adapted every week) depending on their group allocation:

Relevance-condition: “The last two lecture sessions dealt with the planning and interpretation of evaluations. Choose a content (topic, concept, or insight) from your memory of these sessions. Below, write why it may be helpful for your future work as a teacher to know this lecture content. Write approx. four sentences.”

Summary-condition: “The last two lecture sessions dealt with the planning and interpretation of evaluations. Choose a content (topic, concept, or insight) from your memory of these sessions. Below, write a summary of the selected lecture content. Write approx. four sentences.”

The original instructions in German, as well as excerpts from answers that students produced can be found in the online supplement.

5.1.3. Participants

In total, 385 student teachers participated in the course. The final sample consisted of n = 159 students who answered our questionnaire at the end of the course (41.3% response rate; n = 72 in the relevance-condition and n = 87 in the summary-condition). Participants were 71.5% female, 24.1% male and 7 participants identified as non-binary. The two experimental conditions did not differ significantly regarding the distribution of participant’s gender (χ²(2) = .442, p = .80). Participants had a mean age of $M$ = 27.96 years (SD = 5.66) and the two conditions did not differ significantly regarding participant’s age (t(154)  = .243, p = .73). The majority of participants were preparing to teach at secondary school (66.2%), the rest of them was following the track to become an elementary school teacher (33.8%). The two conditions did not differ significantly regarding the distribution of participants’ study track (χ²(1) = 1.869, p = .18). As the intervention was embedded in the knowledge tests and the first three knowledge tests were voluntary, participants received different dosages of the intervention. The majority of the 159 participants included into our final sample had taken part in all four knowledge tests (n = 123; 77.4%), while some participants took part only three times (n = 19, 11.9%), two times (n = 6, 3.8%) or only once (n = 11, 6.9%) and thereby were exposed to lower doses of the intervention. However, there was no significant difference in mean intervention dosage between the two conditions (t(157) = −.182, p = .86). We decided to include all participants who received the intervention prompt at least once, in line with other studies who applied writing interventions (e.g., Hulleman & Harackiewicz, 2009) and who found no dosage effects of such an intervention (Canning et al., 2017)

6. Measures

For all psychological variables, respondents answered on seven-point Likert scales ranging from 1 (does not apply at all) to 7 (fully applies). Students’ course performance score was calculated as the percentage of correct answers on each of the four knowledge tests. A complete overview of all items can be found in the online supplement.

6.1. Perceived relevance

We assessed how relevant students perceived the course contents to be for their future job as a teacher with six items. Two items were more generic (e.g.: “The contents of the lecture are useful for my professional activity as a teacher”) and four items explicitly covered specific application scenarios. These examples had been identified as particularly significant for teachers with experts in the field of DBDM prior to our main study (e.g.: “As a teacher I need theories and methods of pedagogical diagnostic to appropriately evaluate my students.”). Internal consistency was good with Cronbach’s α = .80.

6.2. Self-determined motivation to learn

To develop items that assessed the six different motivational qualities from a SDT perspective, we adapted existing scales (AMS, Vallerand et al., 1993; SIMS, Guay et al., 2000; SRQ-L; Black & Deci, 2000) to the specific setting of a lecture-based university course and to the specific course content, DBDM. To that end, we asked the teaching personnel organizing the course and experts in the field of DBDM to generated possible reasons to engage in DBDM and categorized them according to their degree of self-determination. For example, reasons referring to the exam were moulded into items measuring external regulation, as exams often represent a strong and salient external influence for university students. Or contents referring to DBDM being part of teachers’ professional identity were moulded into items measuring integrated motivation. In this way, we generated two items four each of the six motivational qualities, resulting in six items representing more self-determined forms of motivation (e.g., “I have learned because I will need these contents as a future teacher.”) and six items representing less self-determined forms of motivation (e.g., “I have studied for the final knowledge test, so that I get the confirmation of active participation in the lecture.”). Students were asked to indicate how strongly each of the 12 items applied to them in the course, with the six resulting scores being integrated into a common score, the so-called relative autonomy index (RAI; Grolnick & Ryan, 1987; Howard et al., 2017):

RAI = (intrinsic regulation + integrated regulation + identified regulation)—(introjected regulation + external regulation + amotivation)

This parsimonious and well validated measure allows to compare the differences in the degree of self-determination of learner’s motivation with a single score, with higher scores indicating more self-determined motivation in relation to less self-determined motivation.

6.3. Combined score for internalized extrinsic motivation

To test our assumption that the effect of our intervention on application intentions would be mediated by internalized extrinsic motivation, we collapsed the two items for identified regulation (e.g.,“ I learned for the course so that later I would be knowledgeable in this area.”) and the two items for integrated regulation (e.g.,“ I learned because I am convinced that I need to act based on evidence as a teacher.”) to a subscale with four items, which proved to be strongly reliable with Cronbach’s α = .91.

6.4. Satisfaction of the need for autonomy

Need satisfaction regarding competence and relatedness were also assessed with three items each. As we neither expected nor found an effect of the intervention on the two needs, only results regarding need for autonomy are included. Item content, reliability analyses, and results can be found in the online supplement.

Autonomy satisfaction was assessed with three items. We asked students to rate how much the statements were true, compared to a regular lecture (e.g., “Compared to a typical lecture … I was able to make more of my own decisions regarding learning options.”), Cronbach’s α = .80.

6.5. Knowledge tests

All tasks in the knowledge tests had a multiple-choice answering format, with students having to identify two correct answers out of five options. Each of the first three knowledge tests consisted of 12 multiple-choice questions covering the most recently discussed topic. Six additional questions on each of the course units already completed were added, such that the first test consisted of 12 questions, the second of 18 and the third test of 24 questions. The obligatory fourth knowledge test consisted of 6 questions from each of the four learning units (24 questions).

6.6. Application intentions

To measure students' willingness to apply course contents in their future work as a teacher we developed four items. We consulted experts in the field of DBDM in school contexts and the coordinator of the course about the most probable and useful ways of applying the acquired knowledge in teachers' practice. Four items were formulated, one referring to participation in school evaluations, one to the usage of standardized instruments, one to the consideration of scientific evidence about efficacy of pedagogical measure, and one to the evaluation of own teaching practice (e.g., “As a teacher I intend to evaluate my own lessons with questionnaires.”). Internal consistency was good with Cronbach’s α = .80.

6.7. Additional variables

With the final questionnaire, demographic information (gender, age, and study track) was assessed and a final question asked if students became aware of our experimental manipulation.

6.6.1. Self-efficacy regarding application

We asked students how confident they were regarding the implementation of the skills they acquired within the course when facing obstacles with five items (e.g., “I am confident that as a teacher, I can familiarize myself with the current state of research, even when time is short”). Internal consistency was acceptable with Cronbach’s α = .70.

7. Results

Before we conducted our main analyses, we checked for any influences of gender and study program on our dependent variables. A group of t-tests indicated neither significant differences between male and female students nor between students preparing for secondary school compared to students preparing for elementary school. As a manipulation check, we asked two independent raters to classify all answers students had produced in response to the instructions as either a reflection about relevance or a summary of course contents, with raters being blind to experimental condition. Raters’ classifications correlated with r = .91 and disagreement was resolved by discussion. In all four knowledge tests, answers to the intervention prompt in the relevance-condition were significantly more likely to be rated as a reflection about relevance than answers of participants in the summary-condition (knowledge test 1 χ² = 58.961, p < .001; knowledge test 2 χ² = 66.581, p < .001; knowledge test 3 χ² = 44.697, p < .001; knowledge test 4, χ² = 63.437, p < .001). When asked, only 5% of the participants indicated that they were aware that instructions differed somehow between students.

In our final dataset we had 0.3% of missing data on item level. We ran all analysis on a dataset where missing values were replaced with estimation maximization and as results did not differ we decided to exclude missing values analysis by analysis. To test the effects of the intervention on our dependent measures we used a posttest-only experimental research design (Campbell & Stanley, 1967). We investigated the predicted differences between the two conditions with one-sided independent samples t-tests. We checked homogeneity of variances with Levene’s test and adjusted results if the assumption was violated. To conduct mediation analysis, we used Hayes (2018) PROCESS package version 3.3. We report Cohen’s d with pooled standard deviations as a measure of effect size.

7.1 Validity analyses

To assess validity of our instruments we applied confirmatory factor analysis with maximum likelihood estimation for all our dependent measurements. All models representing the factorial structure of the scales showed indication for acceptable fit (analysis results can be found in the online supplement). We further tested the continuum assumption for motivation to learn, stating that the order of the subtypes of motivation along the continuum of self-determination should be reflected in correlational patterns, with correlations being stronger between neighbouring subtypes and weaker or even negative between subtypes of motivation which are more distant from each other on the continuum (Howard et al., 2017). As can be seen in Table 1, the correlational patterns reflected the continuum structure. Furthermore, in line with SDT, the satisfaction of the psychological need for autonomy as well other adaptive learning outcomes (application intentions and self-efficacy) correlated with the subtypes of motivation, with correlations becoming less negative and then more positive along the continuum.

Table 1. Correlations between different forms of motivation and other dependent variables (n = 159)

		1	2	3	4	5	6	M
1	Amotivation	1	.45**	-.35**	-.62**	-.62**	-.75**	3.90
2	External		1	-.19*	-.40**	-.45**	-.42**	5.25
3	Introjected			1	.58**	.54**	.40**	4.07
4	Identified				1	.83**	.69**	4.05
5	Integrated					1	.68**	4.50
6	Intrinsic						1	2.28
	Application Intentions	-.52**	-.34**	.38**	.64**	.67**	.53**	4.72
	Autonomy Satisfaction	-.36**	-.26**	.19*	.33**	.35**	.41**	4.44
	Self-Efficacy^a	-.37**	-.26**	.30**	.47**	.47**	.37**	4.88

Note: ^a n = 155. * p ≤ .05; ** p ≤ .01.

7.2 Main analyses

Descriptive statistics including means, standard deviations, mean differences, and effect sizes for the six dependent variables are displayed in Table 2.

Table 2. Means and standard deviations, mean differences and effect sizes broken down by experimental condition for all dependent variables

	Summary Condition		Relevance Condition
Variable	M	SD	M	SD	Mean difference	d
Perceived Relevance	4.97	1.04	5.51	0.82	0.53**	0.56
RAI	-4.45	12.09	-0.74	12.09	-3.71*	0.31
Internalized Extrinsic Motivation	4.23	1.34	4.73	1.34	0.50*	0.37
Autonomy Satisfaction	4.11	1.49	4.69	1.39	0.58**	0.40
Test Performance (%)	75.91	15.03	77.32	14.18	1.41	0.10
Application Intentions	4.48	1.21	4.94	0.84	0.47**	0.44

Note. * p ≤ .05. ** p ≤ .01.

7.2.1 Student teachers’ motivation

We found support for our first three hypothesis. Students in the relevance-condition reported perceiving course contents as more useful to their future job, compared to students in the summary-condition, t(157) = −3.53, p < .001, d = 0.56; BCa 95% CI [0.24, 0.88] (hypothesis 1). Participants who were repeatedly asked to reflect on the personal relevance of course contents reported a higher score on the RAI, compared to participants in the summary group, t(157) = −1.93, p = .028, d = 0.31; BCa 95% CI [0.01, 0.62] (hypothesis 2). When looking at internalized extrinsic motivation, thus a combined score of identified regulation and integrated regulation,2 A comparison of the intervention effects on all subtypes of motivation with a MANOVA can be found in the online supplement. Students in the relevance-condition endorsed them to a stronger extent than students in the summary-condition, t(157) = 2.35, p = .02, d = 0.37; BCa 95% CI [0.06, 0.69]. Regarding need satisfaction (hypothesis 3), participants in the relevance-condition reported a higher autonomy in the course compared to participants in the summary-condition, t(157) = −2.52, p = .065, d = 0.40; BCa 95% CI [0.09, 0.72]. We did not find any effects of the intervention on the satisfaction of the needs for competence and relatedness (analyses can be found in the online supplement). Other than expected, we did not find an effect of the intervention on students’ test performance on LPT4, t(157) = −.60, p = .548, d = .10; BCa 95% CI [−.22, .41] (hypothesis 4).

7.2.2 Student teachers’ application intentions

Corroborating hypothesis 5, students who repeatedly reflected about the personal relevance of course contents indicated higher intentions to apply acquired skills and knowledge in the future, compared to students in the summary-condition, t(152.47) = 2.88, p = .003, d = 0.42; BCa 95% CI [0.11, 0.74].

As depicted in Figure 2, mediation analysis indicated an indirect effect of the relevance reflection on application intentions through internalized extrinsic motivation (identified and integrated regulation), b = 0.26, as the bias-corrected 95% confidence interval did not include zero [0.04, 0.52]. The direct pathway between intervention and application intentions was not significant, b = 0.2, p = .11, which indicates a full mediation, meaning that as expected (hypothesis 6), the effects of the intervention on application intentions was fully explained by the indirect path via internalized extrinsic motivation.

Figure 2. Mediation model of the intervention effect on application intention via internalized extrinsic motivation

7.3 Additional analyses

7.3.1 Self-efficacy for application

Students in the relevance-condition reported more confidence in being able to apply course contents against obstacles and challenges, compared to students in the summary-condition, t(157) = −2.19, p = .013, d = 0.35, BCa 95% CI [0.03, 0.67].

8. Discussion

In our study, we applied a brief motivational intervention prompting students enrolled in a master program in teaching to reflect on the relevance of Data-Based Decision-Making (DBDM) for their future professional activity as a teacher. While many countries have embedded DBDM into their teacher preparation programmes, it often is challenging to engage students in the learning contents, as they experience them as particularly difficult and not intrinsically appealing (e.g., Batanero et al., 2011; Heitink et al., 2016; Murtonen, 2005; Murtonen et al., 2008). Also, it is crucial to encourage students to later make use of DBDM in their professional practice, i.e., to systematically gather, analyse and evaluate empirical data with the goal of examining their pedagogical practice, improving instruction in the classroom and evaluating the effectiveness of pedagogical interventions.

As expected, students who had reflected on the relevance of DBDM reported more favourable qualities of motivation, endorsed the relevance of DBDM for their future job to a stronger extent, and felt more autonomous in learning. While they did not improve their knowledge of DBDM, their intentions to engage in DBDM as a future teacher were strengthened. Altogether, results suggest that this short intervention can be implemented in a regular university curriculum without the need of many resources and adaptions and still unfold important motivational effects.

8.1 Promoting motivation to learn

Most students who want to become teachers probably aim to provide high-quality instruction and thereby promote the social and intellectual development of each of their individual future students. However, many teacher students are unaware that basic knowledge of research methods and statistics, the contents of courses about DBDM, might help them achieving these goals. As positive attitudes about the practicability and usefulness of DBDM methods for improving teaching is a crucial factor when it comes to applying them as a teacher, courses about DBDM should convey the content’s importance. Results of the present study suggest that our relevance intervention is a promising tool for this purpose: At the end of the semester students found DBDM to be more relevant for their future teaching if they were regularly prompted to think about the content’s relevance for their future professional activities. This finding is in line with research within the expectancy-value framework (Harackiewicz & Priniski, 2018). The present study complements this line of research by providing evidence that the quality of students’ motivation can be improved by our relevance intervention implemented into a regular university course over an entire semester.

Students who were asked to reflect on the relevance of the course content reported more self-determined motivation to learn, evident in more positive scores on the relative autonomy index (RAI). This suggests that these learners aligned the reasons for learning about DBDM with their own goals as a future teacher and were therefore more sustainably motivated than the students who regularly wrote summaries of the same content. Vansteenkiste et al. (2018) proposed that for a rationale to support internalization, it needs to be attuned to the learner’s perspective. As we asked students to pick the content that they wanted to reflect on and to come up with their own rationale regarding the relevance of the contents, they could attune their arguments to their own values and preferences. This may explain why we found a strong effect of our relevance intervention—in contrast to the findings of the meta-analysis by Steingut et al. (2017) who reported no overall effect of external rationale provision on self-determined motivation.

The effect of our intervention on autonomy satisfaction indicates that students in the relevance condition felt more self-determined within the course. Although the learning conditions and requirements concerning the knowledge tests were identical for students in the two experimental conditions, those who participated in the relevance intervention experienced themselves as more autonomous. This suggests that feelings of autonomy can be enhanced simply by asking learners to find their own rational why the subject matter is important. The effect was comparable in size to what other studies have found. However, so far, most studies have only examined the effect of relevance interventions combined with other autonomy-supportive practices (Steingut et al., 2017).

The effect of our relevance intervention was particularly evident in a subgroup of self-determined forms of motivation, namely internalized extrinsic motivation, the combination of identified and integrated motivation. Students in the relevance condition were more likely to report that they learned because they needed the knowledge later as a teacher and it was part of who they wanted to be as a teacher.

Contrary to our expectation, students having reflected on the relevance of course contents did not perform better in the knowledge test. This might be due to the pathway from motivation to achievement being constrained by the learners’ inability to translate high quality motivation into high quality learning or by the fact that students had to meet external requirements (pass the exam, meet the deadlines) such that differences in the quality of their motivation did not play out in differences in test performance (Vu et al., 2021). Apparently, then, the intervention had no effect on students working and learning behaviour. One possible explanation is that performance in the last knowledge test was the criterion for passing the course. This external incentive could have resulted in students from the summary-condition being highly motivated to learn as well—albeit for the test. The learning motivation of the two groups therefore differed more in terms of quality than in terms of intensity, with the motivation of the relevance intervention group being more sustainable.

8.2 Promoting application intentions

The application of DBDM skills in school enables teachers to adapt and to develop their instruction and to continuously align their professional activities with the advancing scientific knowledge—but it is also associated with an additional amount of work. Administrative measures to encourage teachers to undertake this effort, e.g., performance-based pay, do not always have the desired effects (United Nations Educational, Scientific and Cultural Organization, 2020). It is therefore even more important that teachers develop a self-determined motivation to use DBDM. The current study suggests that the foundations for this motivation can already be laid during teacher training. As predicted, students who thought about why DBDM was relevant for their future job reported higher intentions to apply DBDM (e.g., evaluate own lessons, participate in school evaluations) later, compared to students who wrote summaries. We found that this difference was fully mediated by internalized extrinsic motivation. This suggests that the increase in application intentions was caused by students in the relevance condition having aligned the learning objectives of the course with their own goals and values. This finding is in accordance with Hagger and Chatzisarantis (2016) who found that self-determined motivation in class predicted out-of-class behavioural intentions. In our study, we only measured the intention to use the course content (DBDM) and not the actual application. However, Hagger and Chatzisarantis (2016) found a substantial effect of intentions on actual behaviour.

Our relevance intervention not only strengthened students’ application intentions for DBDM but also their self-efficacy regarding their implementation. Students who were asked to think about the relevance of DBDM saw themselves as more able to apply their knowledge even in the face of obstacles and difficulties (e.g., time pressure, complex situations). Assuming that this effect is due to the autonomy-supportive character of the intervention, the effect size is in line with results of Hagger and Chatzisarantis (2016) on perceived behaviour control. Similarly, Ross et al. (2016) found that self-determined motivation to learn was positively related to university students’ self-efficacy regarding information literacy.

8.3 Limitations and future directions

The present study tested a relevance intervention in a teacher training course on DBDM. Even though we applied a randomized controlled design, it would be possible to rule out any potential baseline differences between groups in upcoming experiments.

Future research might investigate whether comparable effects can also be found with other course content and in other disciplines and test if the promising effects last and also translate to difference in application behavior.

In our study students had to pass an exam at the end of the semester. Accordingly, the motivational effects of the relevance intervention were observed within a setting in which there was also an external incentive to learn. It remains to be tested what motivational effects a relevance reflection has without the concurring influence of external incentives and whether it may elicit sufficient motivation and knowledge acquisition to even avoid mandatory tests.

Future studies could combine the relevance intervention with other autonomy-supportive practices, e.g., the use of non-controlling language, to explore whether the effects are amplified and also become visible in learning behaviour and performance. The effects of our intervention were observed on self-report data only. Behavioural data, e.g., the use of DBDM skills in schools, would further substantiate the effectiveness of the intervention.

We will gladly provide our research materials, like the intervention instructions and the motivation scales tailored to the DBDM-content, to others who wish to research these or related questions.

Ethical approval

This study received approval by the ethical committee of Freie Universität Berlin (032/2019).

Conflicts of interest/Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Short summary of key research activities

We aim at fostering more self-determined qualities of motivation and self-regulated learning in courses on data-based decision-making (DBDM) which does not appeal attractive to many teacher students but is nevertheless of essential importance in their future profession. In an experimental study, we had students reflect about the relevance of DBDM and succeeded in strengthening their motivation to apply it in their future profession as a teacher. We further conducting experiments regarding the language of formative feedback to support self-determined motivation and self-regulated learning in students.

Correction

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Acknowledgements

The authors would like to express their thanks to Malte Roswag, Bettina Hannover and Alexa Banker who provided valuable assistance during the research.

Supplementry material

Supplemental data for this article can be accessed here.

Additional information

Funding

No funds, grants, or other support was received