Connecting explanations to representations: benefits of highlighting techniques in tutorial videos on students’ learning in organic chemistry

ABSTRACT

Research in science education agrees that one of the key challenges of learners in the discipline is certainly connecting domain-specific representations to the underlying concepts. One way of supporting students to make applicable connections is using purposefully designed highlighting techniques in multimedia instructions. In order to examine the influence of different highlighting techniques on learning, 171 chemistry undergraduate students were provided with tutorial videos either with static, dynamic or without highlighting. The results show that students viewing tutorial videos with dynamic highlighting gave more sophisticated answers in direct retention tasks. Furthermore, results indicate that low prior knowledge is compensated by both static and dynamic highlighting techniques. This finding is supported by causal mediation analysis, which indicates that the effect of prior knowledge is moderated by the different highlighting techniques. Besides student learning outcomes, students’ evaluation of the different tutorial videos shows significant benefits of the highlighted instructions in terms of perceiving higher comprehensibility. The results support the use of appropriate highlighting techniques in instructional formats to foster a stronger link between conceptual knowledge and representations.

KEYWORDS:

• Tutorial videos
• representations
• highlighting

1. Introduction

The domain of chemistry relies on a system of external visual representations to display imperceptible phenomena on the molecular level. Using and understanding these types of representations is one of the main requirements in chemistry (Cooper et al., 2013; Gilbert & Treagust, 2009; Stieff et al., 2011). Particularly in organic chemistry, understanding the symbolic language is essential to infer chemical concepts from these representations and to use them as ‘evidence to support claims, draw inferences and make predictions’ (Kozma & Russell, 1997). Working with representations includes the ability to analyse specific features, generate new representations, and distinguish which representations can support specific arguments or not, among others (Kozma & Russell, 2005; Stieff et al., 2011). To develop this skill-set, learners need to connect chemical concepts that are encoded in a representation to visible surface features (Kozma, 2020; Rau, 2017). However, according to the representation dilemma, representations pose a twofold demand: on the one hand, students need to learn new information from representations, on the other hand, they might not yet fully understand the representations themselves (Rau, 2017). This dilemma might be exacerbated from the circumstance that representations are often taught as a mere ‘by-product’ of other topics and are therefore not explicitly discussed (Hilton & Nichols, 2011; Popova & Jones, 2021; Taskin & Bernholt, 2014). Therefore, it might not be surprising that dealing with representations is considered a major challenge for students (Taber, 2009; Taskin & Bernholt, 2014).

In chemistry education research, students’ difficulties with understanding these representations are well known (Cooper et al., 2013; Gilbert & Treagust, 2009; Johnstone, 1982). There is a consensus in the literature that learners often tend to either rely on surface features of representations in order to solve problems, or to use heuristics that involve the application of symbolic rules grounded in memorisation or intuition (Graulich, 2015b; Maeyer & Talanquer, 2013; Weinrich & Talanquer, 2015). Instead of connecting the appropriate meaning to the symbols, students are known to take a connect-the-dots or decorating-with-arrows approach while overall prioritising form over function (Anzovino & Bretz, 2015; Bhattacharyya & Bodner, 2005; Ferguson & Bodner, 2008; Grove et al., 2012; Rappoport & Ashkenazi, 2008; Strickland et al., 2010). Not only the ability to incorporate mechanistic reasoning into argumentation is described as a rather slow development (Weinrich & Talanquer, 2015), but also the application of heuristics seems to even hinder students to reach more sophisticated levels (Cooper et al., 2013).

While students’ problems and deficits in learning organic chemistry are well grounded in the literature, less is known about ways to foster the ability to infer chemical concepts from representations.

2. Theoretical background

Representational competencies can be considered an umbrella term for an essential set of knowledge and skills constituting domain-specific expertise (Daniel et al., 2018; Rau, 2020; Stieff et al., 2011; Tippett, 2016; Treagust et al., 2003). While serving as a bridge construct between cognitive and situative orientations, practices associated with components of representational competencies (e.g. analysing features of a representation) are well-grounded in multimedia learning theories (Rau, 2017). With regard to developing representational competencies, the three most acknowledged multimedia learning theories (Design, Function and Task Framework (Ainsworth, 2006), the Cognitive Theory of Multimedia Learning (Mayer, 2014) and the Integrated Model of Text and Picture Comprehension (Schnotz, 2014)) agree that learning from representations requires visual understanding of how information is encoded in those representations. Visual understanding involves three consecutive processes: selecting meaningful visual features for further processing, organising these visual features into an internal representation, and integrating the internal representation by mapping it to prior domain knowledge (Mayer, 2014; Schnotz, 2014). Thus, an external chemical representation demands the learner to identify relevant features (e.g. the symbol ‘C=O’ in a Lewis-structure), then to organise the selected information to the referent that is internally represented (e.g. mapping the ‘C=O’ to a carbonyl group) and to integrate the internal representation with conceptual knowledge (e.g. mapping carbonyl to its chemical properties, such as a negative inductive effect). Increasing expertise allows learners to efficiently and effortlessly form internal representations based on recognition, which involves fluent retrieval of conceptual knowledge (Chi et al., 1989; Kellman & Massey, 2013; Schnotz, 2014).

While experts see ‘at a glance’ what an external representation depicts, visual understanding is facilitated when the connection between representation and conceptual knowledge is made explicit (Chi et al., 1989; Hilton & Nichols, 2011; Kellman & Massey, 2013; Rau, 2020). To foster students’ visual understanding, evidence from cognitive sciences suggests that scaffolding with a signal or highlighting technique might be an appropriate strategy to overcome the representation dilemma (Rau, 2017; van Gog, 2014). By using highlighting in digital instructional formats, learners’ attention is purposefully guided towards the relevant parts of the representation to accentuate the organisation of the material and to reduce search space (Boucheix & Lowe, 2010; van Gog, 2014). When encountering new learning materials, these highlights are particularly helpful by facilitating the process of information selection (Mayer, 2014).

Common chemistry textbooks make use of highlighting by colouring individual parts of a representation such as functional groups (Bruice, 2016; Clayden et al., 2012). This static highlighting intends to guide the readers to specific parts of the representation or to the ideas discussed in the text. Bhattacharyya and Bodner (2005) report that this kind of highlighting in organic chemistry textbooks appears to be helpful to students to the extent that they experience difficulties when transitioning to textbooks that no longer have such cues.

Nevertheless, textbooks are limited to presenting written explanations and corresponding visual representations sequentially, meaning that multiple sources of information must be integrated mentally before they can be understood (Mayer, 2014; Schnotz, 2014). By presenting new information in this manner, only the channel of visual processing is addressed, and thus attention must be split between explanation (text) and representation, which places an additional demand on the learner (Ayres & Sweller, 2014). According to the modality principle, addressing two sensory channels (auditory and visual) results in superior learning (Low & Sweller, 2014; Tindall-Ford et al., 1997). This increase of learning occurs when new information is explained by audible narration rather than by on-screen text, as is the case of tutorial videos. It was shown that the modality principle is especially efficient when the learning material is complex, as is the case of reaction mechanisms (Tindall-Ford et al., 1997). Following this line of argumentation and using tutorial videos for instruction, a permanent (static) colour-coded highlighting might not adequately account for the dynamic nature of an (audible) explanation. To address this, one approach is to combine the verbal explanations with highlighting by colouring parts of a representation sequentially while they are being referred to. These dynamic highlights are synchronised spatially and timely with the explanation. Especially in learning environments that address complex learning materials, dynamic highlighting aligned to verbal explanations were found to result in superior learning (Jamet et al., 2008; Jeung et al., 1997; Tabbers et al., 2004), as they facilitate learners to attend to information in a recommended order and to distribute their attention accordingly by a shortened search time (Boucheix & Lowe, 2010; de Koning et al., 2009).

3. Research questions

When learning organic chemistry, students’ main hindrances seem to be the ability to connect representations with underlying chemical concepts (Graulich, 2015a). A potential approach to foster this ability is incorporating specific highlighting into instructional material. Therefore, our study aims to provide evidence that purposefully designed highlighting techniques (either static or dynamic) in tutorial videos support students’ visual understanding, e.g. to make appropriate inferences about implicit chemical concepts from representations of organic reaction mechanisms. Accordingly, students’ learning gains and perception when working with these tutorial videos were analysed in order to answer the following research questions:

• RQ 1: Do tutorial videos with static or dynamic highlighting of relevant parts of a representation improve students’ visual understanding in the context of reaction mechanisms more than tutorial videos without highlighting?

We hypothesise, based on the existing literature, that static highlighting results in beneficial learning compared to the non-highlighted condition, and, that dynamic highlighting results in substantial learning gains compared to the static and control condition (Jamet et al., 2008; Jeung et al., 1997; Tabbers et al., 2004).

Besides the pre- and post-tests measuring different facets of visual understanding (e.g. information retrieval from structural formula), the tutorial videos were accompanied by a set of immediate retention tasks that directly address aspects and relations that are explicitly mentioned in the videos (c.f. Method section). This measure indicates to what degree students elaborate on or rephrase the main arguments presented in the video. Based on these three measures, we intend to address the following research question:

• RQ 2: Does students’ retention when watching the videos mediate the effect of their prior knowledge on their knowledge gain?

• RQ 3: Do different highlighting techniques moderate the mediation of prior knowledge on learning gain via retention?

We assume that the relation between students’ prior knowledge and their knowledge after the intervention is at least partially mediated by the degree to which students are able to rephrase the main arguments in the videos (i.e. retention tasks). As students’ retention is intended to be guided by the different highlighting techniques implemented in the treatment conditions, we further expect this mediation to be moderated by the three intervention groups.

In addition to objective measurements of knowledge and learning gains, we were also interested in students’ perception of the different highlighting techniques and whether these techniques influence their subjective ratings of the comprehensibility of the tutorial videos.

• RQ 4: Does highlighting influence students’ perception of the tutorial videos’ comprehensibility?

In line with our expectations regarding the benefit of the different highlighting techniques on students’ learning gains, we assume that students perceive the highlighting to be helpful to process and understand the content covered in the tutorial videos. Accordingly, we expect students to rate the comprehensibility increasingly higher when comparing groups from control to static to dynamic.

4. Method

In order to examine the effect of the different highlighting techniques, the format of tutorial videos was chosen as the conveyed content could be kept constant. At the same time, the different highlighting versions could be implemented. The purposefully designed tutorial videos were presented to students in a randomised controlled intervention (Figure 1).

Figure 1. Overview of the research design, measures and procedure. No highlighting is shown in the control group, functional groups are colour-coded in the static group, the translucent red dot moves synchronously to the explanation in the dynamic group.

4.1. Participants and setting

For this study, 198 students from beginner organic chemistry courses at two German universities were recruited on a voluntary basis. The organic chemistry lectures of the two testing locations were compared prior to the study and were very similar in curriculum and teaching style. At the time of the intervention, nucleophilic substitution reactions were covered in the lectures, however, these courses followed a rather traditional teaching approach based on name-reactions and predict-the-product tasks. Thus, students were unfamiliar with the task format presented in the videos. We chose students from this course as appropriate addressees for our intervention since participation in this course provided sufficient prior knowledge to potentially understand the representations displayed in the tutorial videos. Informed consent was obtained from all participants after the provision of information about data-handling and privacy protection rights. Also, students were informed that participation would not affect their course grade and that they could opt out from the study at any time without consequences. For mapping pre- and post-tests, pseudonyms were used. The research design followed ethical standards compliant with the German Research Foundation. 27 participants were excluded due to missing group affiliation, un-assignable pre- and post-tests or overall missing data. Consequently, data from 171 students were analysed.

The participants were randomly assigned to one of the experimental conditions of the between-subject design (cf. Figure 1, cf. section instruments). The sample composition within the three groups was comparable (Table 1).

Table 1. Group composition and demographic statistics.

Participants		Control group	Static group	Dynamic group
		56	59	56
Gender*	Female	48%	66%	54%
	Male	52%	34%	46%
Mean Abitur grade**	M (SD)	2.01 (.57)	2.12 (.62)	1.96 (.56)
Cognitive ability test	M (SD)	17.17 (4.15)	16.79 (4.31)	16.22 (4.56)

*No other genders were reported by the participants.

**In Germany, the Abitur is equivalent to a high school diploma. Smaller numbers indicate better grades, ranging from 1 (very good) to 4 (sufficient).

4.2. Instruments

4.2.1. Tutorial videos

The central element of the intervention are the tutorial videos. Overall, three videos with a length of approximately 5 min each were developed. The development and design of the videos are described with regard to three main aspects: content, task format, and explanation. The content of the videos covers nucleophilic substitution reactions, which are exemplary to basic reaction mechanisms in introductory organic chemistry at the university level. Influential factors in nucleophilic substitution that determine reactivity are leaving group ability, nucleophilicity, and substrate effects. These factors are governed by implicit properties of the reactants and students need to connect those properties to the explicit features of the structural formula at hand to make a claim about the reaction process.

We chose case comparisons as the task format (Figure 2). Rooting in analogy learning, case comparisons have been a long established pedagogical tool across various disciplines (e.g. Gentner, 1989; Treagust et al., 1992; see Alfieri et al., 2013, for a meta-analytic review). This format is also known in chemistry education to be advantageous in terms of eliciting students’ reasoning of chemical mechanisms by engaging them to compare the (explicit) structures and to critically weigh the (implicit) factors that influence the reaction process (Bodé et al., 2019; Caspari et al., 2018). Case comparisons typically display two similar reactions, which should be compared and discussed by the learner, e.g. to determine the reactivity based on commonalities and differences between the two cases (Graulich & Schween, 2018). When faced with this task format, beginner students have a significantly lower proportion of comparative eye-movements than advanced students (Rodemer et al., 2020). This finding indicates that case comparisons with visual highlights might support specifically students with low prior knowledge.

Figure 2. Example case comparison task from the tutorial videos.

The verbal explanation given in the tutorial videos is based on the principles of the comparative mechanistic reasoning framework and provides a complete cause–effect description (Caspari et al., 2018). The explanations follow a step-by-step structure: A brief overview addressing the changes that occur in both reactions (I), naming the goal of the comparison (II), emphasising the visual commonalities and differences (III), stating the relation between reactivity and influential factors (IV), explaining and comparing the influences of the implicit properties (V), summarising the effects on the transition state to refer to the energetic proceeding of the reactions (VI), and, finally, coming to a justified statement about the reactivity of both reactions (VII). Subsequently, all explanations were reviewed and approved by an independent expert of organic chemistry.

Following the research questions, the three tutorial videos are provided with either no cues (‘control’), permanent colour-coding of functional groups (‘static’), or sequential highlighting (in form of a translucent red dot), synchronised with the verbal explanation (‘dynamic’), while the verbal explanation and the permanent display of representations remain the same in all three conditions (Figure 1; for original (German) and translated videos, see https://osf.io/r4sx3/?view_only=84f3319508594da7b1d5d93caf942c3f).

4.2.2. Cognitive abilities test

The subscale ‘figural analogies’ (N2) from the cognitive abilities test (KFT-R; Heller & Perleth, 2000) was used. In 25 items, a pair of two figures combined with a third figure is presented. Students are required to select one figure out of a list of five other figures that matches the third in such a way that both pairs illustrate an analogous relation. The instrument is a speeded test (8 min) and a well-established and standardised instrument to assess abstract-logical and perception-bound aspects of intelligent behaviour. The reliability of the test was sufficiently high (Cronbach’s α = .79).

4.2.3. Pre-/post-test

A test with 34 items was developed to assess the learners’ visual understanding, e.g. the ability to make appropriate inferences about implicit chemical concepts from representations, with respect to nucleophilic substitution reactions in organic chemistry. The purpose of the test was the assessment of students’ knowledge gains based on the intervention. The test draft was reviewed by an organic chemistry lecturer and two experts of chemistry education. Based on their recommendations, some items were revised and refinements were made. A pilot testing included 21 students. Students’ answers to the test and feedback resulted in further refinement of wording and revision of some items.

The items reflect tasks that are typical for representational competencies and, hence, visual understanding, e.g. selecting information from representations and meaningfully organise them (Rau, 2017). Six different aspects of nucleophilic substitution reactions were addressed in the items: information-retrieval from structural formula (12 items), case comparisons of functional group properties (11 items), case comparisons of reactivity between two similar reactions (3 items), predict-the-product tasks (3 items), sorting tasks for leaving group ability (3 items), and cloze texts on reaction process description (2 items). Answer formats included multiple-choice, true-false, constructed response, cloze texts, and drawings (for exemplary items cf. supplementary material S3).

For the analysis, each item was scored binary and an overall mean test score by participant was calculated. In accordance with recent studies (Scheid et al., 2018), the specific processes of visual understanding covered in the different tasks are not assumed to reflect individually measurable subskills, but the test is assumed to measure a one-dimensional construct. Test score reliability was assessed by calculating Cronbach’s alpha and was found to be sufficient (pre-test: α = .79; post-test: α = .68).

4.2.4. Retention tasks

Immediately after each tutorial video, two retention tasks concerning the chemical factors that influence the reaction speed were presented to students. These factors were addressed in the videos; therefore, the retention tasks measure the elaborateness of the students’ answers in rephrasing the main arguments. One example task was: ‘Please explain the faster bond formation in reaction B’.

The theoretical considerations that went into these tasks and the tutorial videos are based on prior research concerning case comparisons in organic chemistry (Caspari et al., 2018; Graulich & Schween, 2018). The level of elaboration was scored using a 4-point rating scale ranging from 0 (wrong answer) to 3 (emerging cause–effect relation) (Figure 3). Inter-rater reliability with two coders was determined by calculating Cohen’s kappa for coding one fourth of the overall answers (n = 43), which resulted in κ = .79. Refinements were made after resolving divergent codes. Since the inter-rater reliability was good, only one rater continued scoring the remaining answers. The scores of the retention tasks were averaged into a total retention score which was used for the later analysis. Again, score reliability for the retention tasks was found to be sufficiently high, α = .63.

Figure 3 Scoring of retention tasks.

4.2.5. Evaluation questions

Between the three videos, participants were asked to answer four evaluation questions on a 4-point Likert scale (1 – ‘totally disagree’ to 4 – ‘totally agree’). These were, for example: ‘The presentation in the video was easy to understand’ (see Appendix for all items). Test score reliability was calculated using Cronbach’s alpha for each of the three videos. In all three videos, the second item was not correlated with the remaining three items. Thus, this item was dropped, and a scale consistent of three items was formed, resulting in α_Video1 = .74, α_Video2 = .77, α_Video3 = .79. According to the conceptual considerations that went into the item design, the scale covers students’ subjective rating of the comprehensibility of the videos.

4.3. Data collection

The intervention was conducted in a single session of 1½ hour in the chemistry department’s lecture halls. Participants were assigned randomly to the experimental conditions while the size of the groups was balanced. First, the purpose of the study and information about legal rights and privacy protection were presented. After providing informed consent by signature, participants filled in a short questionnaire on demographic data. Afterwards, the cognitive abilities test (8 min) and the pre-test (25 min) were conducted. Then, the intervention started. Participants used either tablets or laptops with headphones to watch the tutorial videos. Technical support was provided when necessary. After each of the three videos, students were asked to answer the retention tasks and the evaluation questions before continuing with the next video. Overall, the intervention lasted about 30 min. The intervention was then followed by the post-test (25 min).

4.4. Analytical strategy

Following our research questions, several statistical procedures were conducted. To determine differences in the treatment groups, analyses of variance and subsequent post-hoc comparisons were conducted. If test assumptions (e.g. normal distribution) were violated, robust procedures were used instead (Field & Wilcox, 2017). For all statistical analyses, an α-level of .05 was used. As a measure for the effect size, (partial) η² is reported, where we qualify values of .01 as small, values of .06 as medium, and values of .14 as large effects (Cohen, 1988). In the case of pairwise comparisons, we report the correlation coefficient r as effect size, where values are qualified as small (.1 ≤ .3), medium (.3 ≤ .5) or large (≥.5) (Cohen, 1988). For robust procedures, we chose heteroscedastic one-way ANOVAs for trimmed means including the explanatory measure of effect sizes ξ, with the same ranges for interpretation as r (Mair & Wilcox, 2020).

In the case of robust repeated measurement methods, we used 2000 bootstrap samples and a default mean trim of 20%. Post-hoc tests are applied with 2000 bootstrap samples as well. To determine the effect of the different groups on the post-test while controlling for the pre-test score, estimated marginal means were calculated. For analyses of covariance, robust linear models with continuous predictors were chosen (Field & Wilcox, 2017).

To follow-up on possible significant findings for learning outcomes, moderated mediation models were constructed to test whether a possible relation between group, pre- and post-tests might be mediated by the retention tasks (Tingley et al., 2014). Unstandardised indirect effects were computed for each of 2000 bootstrapped samples, and the 95% confidence interval was computed by determining the indirect effects at the 2.5th and 97.5th percentiles.

Analyses were conducted in R 4.0.2 with packages tidyverse (Wickham et al., 2019), robustbase (Maechler et al., 2020), ggeffects (Lüdecke, 2018), WRS2 (Mair & Wilcox, 2020) and mediation (Tingley et al., 2014), among others.

5. Results

5.1 . Description of pre-intervention baseline and control of group randomisation

In order to analyse the randomisation to the three intervention groups, students’ scores in the pre-test, the cognitive abilities test, and their high school diploma (Abitur) grade were compared to ensure comparable groups prior to the intervention. With regard to the pre-test, neither group differ in their performance, F(2,168) = 0.91, p = .404, η² = .011. Also, the Abitur grades, F(2,166) = 1.17, p = .313, η² = .014, as well as the cognitive abilities scores are comparable between groups, F(2,168) = 0.56, p = .574, η² = .007. These findings indicate comparable conditions for all groups and that the randomisation was apparently successful.

5.2. Treatment effects on students’ retention of the tutorial videos

As for the retention tasks, an ANOVA was performed to determine differences between the treatments. The groups differed significantly from each other in the elaborateness of their answers, F(2,168) = 10.75, p < .001, η² = .11. In pairwise comparisons with Bonferroni-adjusted p-values, the control group (M = 2.26) and the static group (M = 2.41) do not differ significantly, t(168) = 2.14, p = .102, r = .16. However, the dynamic group (M = 2.59) has significantly higher mean scores than the control group, t(168) = 4.63, p < .001, r = .34, as well as the static group, t(168) = 2.55, p = .035, r = .19 (Figure 4).

Figure 4. Retention task scores by group. Points indicate mean-values, error bars 95% confidence intervals. Results of pairwise comparisons are indicated by significance level (NS. p > .05; *p < .05; **p < .01; ***p < .001).

5.3. Treatment effects on students’ knowledge gains

To determine differences in the post-test score by highlighting technique, a robust heteroscedastic one-way ANOVA was performed, since the test assumption of homogeneity of variance was violated. Based on this analysis, the groups reached comparable scores that do not differ significantly, F_t = 1.24, p = .296, ξ = .20 [0.04; 0.40]. Further, estimated marginal means were calculated to determine the effect of the different groups on the post-test while controlling for the pre-test score. After adjustment for the pre-test score, no statistically significant difference in the post-test score between the groups was found (Figure 5).

Figure 5. Estimated marginal means of the post-test scores after controlling for the pre-test by group. Points indicate mean-values, error bars indicate 95% confidence intervals.

With regard to learning gains from pre to post, a robust repeated measurement analysis of variance was performed, since Levene’s Test indicated heterogeneity of variances. Results suggest a significant increase on the test performance from pre to post measurement across the three groups, F_t = 36.90, F_crit = 3.57, p < .05 (Figure 6). Post-hoc tests confirm the significant increase of the trimmed means in each group, $\hat{\psi }=0.06\left[0.04,0.08\right]$.

Figure 6. Test scores from pre to post measurement by group. Points indicate mean-values, error bars indicate 95% confidence intervals.

In the next step of our analysis, we added the pre-test and the retention task scores as covariates to a robust linear model, as we assumed possible interaction effects. Since knowledge of 0 is not an interpretable value, the predictor variables were scaled to a mean of 0 and a SD of 1. Consequently, the reported coefficients (standardised β) can be interpreted in such a way that they reflect the effect of a pre-test score (or a retention score, respectively) that is one standard deviation above the average of the present sample on the post-test score. Also, group membership was dummy coded to compare both the static and the dynamic group to the control group.

The overall model had a large explained variance of R² = .56 (cf. Table 2). In line with the previous steps of the analysis, no main effect of both the static and the dynamic group was found. However, significant interaction effects were found between the pre-test score and both the static and the dynamic group. Both significant interactions have negative coefficients, which suggests a decrease of the positive effects of the pre-test on the post-test in these two treatment groups (Figure 7). While the control group participants’ post-test score depends mainly on their pre-test performance (as reflected by the higher slope in Figure 6), the relation between pre-test and post-test scores is weakened by the two highlighting techniques.

Figure 7. Interaction effects of the static and dynamic groups with the pre-test score. Error bars indicate 95% confidence intervals.

Table 2. Coefficients of the robust linear model.

Variable	Std. β	SE	t	p
Pre-test	.09	.01	9.61	< .001
Group static	.01	.01	0.46	.64
Group dynamic	.01	.01	1.02	.31
Retention	.01	.01	1.27	.21
Pre*static	−.04	.01	−2.78	.006
Pre*dynamic	−.03	.01	−2.49	.013
Static*retention.	.02	.01	1.55	.12
Dynamic*retention.	−.01	.02	−0.74	.46

Significant p-values are displayed in bold.

To further inspect the interaction effect of a weakened relation between pre- and post-test in the treatment groups, it was investigated how participants with either high or low prior knowledge scored from pre to post measurement and whether treatment groups differ from each other. Prior knowledge groups were formed based on students’ pre-test performance. Students with scores higher than one standard deviation from the overall mean were assigned to the high prior knowledge group and students below one standard deviation from the overall mean were assigned to the low prior knowledge group. Two ANOVAs were executed to uncover differences in the post-test performance by treatment (Figure 8). In the case of high prior knowledge students, no differences in the post-test were found, F(2,30) = 0.80, p = .457, η² = .051. For low prior knowledge students, a significant difference is found, F(2,26) = 3.59, p = .042, η² = .216. Pairwise comparisons with Bonferroni-adjusted p-value show significantly higher post-test scores for the dynamic group compared to the control group (t(26) = 2.63, p = .042, r = .458), but no differences between control and static (t(26) = 1.45, p = .478, r = .273) or static and dynamic (t(26) = .76, p = 1, r = .147). Therefore, the results indicate that low prior knowledge is compensated by the corresponding highlighting while these techniques do not hinder high prior knowledge students.

Figure 8. Test scores from pre to post measurement, separated into high prior knowledge students (left) and low prior knowledge students (right) by group. Points indicate mean-values, error bars indicate 95% confidence intervals.

5.4. Moderated mediation analysis

With regard to the temporal course of the data collection (cf. Section 5), we investigated whether the relation between students’ prior knowledge (pre-test) and their knowledge after the intervention (post-test) is mediated by the elaborateness of students’ answers in the retention tasks following each video. Further, we investigated whether this mediation is moderated by the different highlighting techniques in the intervention groups (Figure 9). The indirect effect was calculated by multiplying the effect of the pre-test on the retention test and the effect of the retention test on the post-test. Further, the significance of this indirect effect was tested using bootstrapping procedures. As depicted in Table 3, the regression coefficients between pre-test and post-test and the regression coefficients between retention test and post-test were significant in all three groups. The bootstrapped unstandardised indirect effect was between .037 (control group), .040 (static group), and .050 (dynamic group). As zero is not included in the confidence intervals for the static und dynamic group, it can be concluded that the indirect effect was statistically significant. This does not apply to the control group since the confidence interval ranges from −.002 to .10. Thus, the effect of the pre-test score on the post-test score was partially mediated via the retention test score and these relations were moderated only by the static and dynamic groups. The sequence of results (control < static < dynamic) is in line with the theoretical considerations that went into the study design.

Figure 9. Moderated mediation model for the relations between pre- and post-test mediated through the retention tasks and moderated by the groups.

Table 3. Coefficients of moderated mediation model.

Group	Average causal mediation effect [95% CI]		Average direct effect (β1^a)	Total effect	Proportion mediated	β2^a	β3^a
Control	.037	[−.002, .10]	.61***	.65***	.05	.215**	.171**
Static	.040*	[.004, .09]	.41***	.45***	.08*	.234*
Dynamic	.050*	[.007, .10]	.34***	.39***	.12**	.292***

Note: ***p < .001, **p < .01, *p < .05

^a For the interpretation of β coefficients cf. Figure 9.

5.5. Students’ perception of the tutorial videos

The last part of the analysis focuses on the evaluation questions after watching the tutorial videos. To determine differences in the perceived comprehensibility across groups for each video, a repeated measurement ANOVA was performed (Figure 10). While the main effect for the video was neglectable, F(2,340) = 18.19, p < .001, η² = .01, the main effect for groups was medium to large, F(2,168) = 14,97, p < .001, η² = .13. To analyse group differences for each video, pairwise comparisons with Bonferroni adjustment were conducted. For video 1, an increase of students’ perceived comprehensibility was observed from the control to the static group (p = .002, r = .26) and from the control to the dynamic group (p < .001, r = .39). No difference was found between the static and the dynamic group (p = .102). In video 2, significant differences between control and dynamic (p < .001, r = .31), but not between control and static (p = .087) and static and dynamic (p = .134) were found. In video 3, pairwise comparisons show significant differences between control and static (p = .002, r = .26), as well as between control and dynamic (p < .001, r = .39), but not between static and dynamic (p = .116). Overall, a similar pattern of results was found in all three videos, albeit with slight differences in the magnitude of the effects of the three treatment conditions on students’ perceived comprehensibility.

Figure 10. Evaluation scale comprehensibility by group across videos. Points indicate mean-values, error bars indicate 95% confidence intervals.

6. Discussion

The main goal of this study was to investigate if purposefully designed highlighting techniques (either static or dynamic) in tutorial videos support students’ visual understanding of chemical representations by explicitly connecting underlying concepts to representations used in organic reaction mechanisms. To compare effects, a randomised controlled trial with three groups (control, static, dynamic) was conducted. Based on comparable pre-test scores, cognitive abilities, and high school diploma (Abitur) grades, the randomisation was successful.

The results of the immediate retention tasks after each tutorial video suggest that a tutorial video with dynamic highlighting might be more beneficial to the learner than the provision of a tutorial video with static or no highlighting. As the retention tasks prompted students to re-narrate the content of the given videos, differences in scores reflect the amount of information that students extracted while watching the videos (cf. Figure 4). Dynamic highlighting might have facilitated the process of selecting information and organising it internally (cf. Mayer, 2014; Schnotz, 2014). Following the multimedia principles of modality and signalling, students’ attention might be constantly navigated towards the relevant features of the representations due to the synchronicity of the verbal explanation and the dynamic highlighting (de Koning et al., 2009; Jamet et al., 2008; Low & Sweller, 2014; Tabbers et al., 2004; van Gog, 2014). This hypothesis is supported by previous studies, which document that the contiguous processing of a representation is supported by visual cues that draw the participants’ attention to the relevant elements (c.f. Boucheix & Lowe, 2010; de Koning et al., 2009).

In contrast to the effects of highlighting on direct retention tasks, no main effects of the treatment groups were found on students’ visual understanding in the post-test. Nonetheless, our data indicates significant learning gains from pre to post, i.e. students performed better after the intervention, but indistinguishably better between groups. This might be due to the short intervention duration (½ hour), in which the benefit of the different highlighting techniques may be too little to be sustainably and differentially effective. Also, the effect of the highlighting compared to the verbal explanation might be substantially smaller.

However, one can assume that highlighting might not be equally beneficial for all students. Our data suggests that the effect of learners’ prior knowledge on their learning gains differs between the highlighting conditions, as indicated by significant interaction effects between the pre-test and the two treatments (cf. Figure 7). This finding is supported by pre-/post-comparisons for high and low prior knowledge students. While no significant differences in the post-test could be found for high prior knowledge students, in the case of low prior knowledge students, the dynamic group outperformed the control group significantly (cf. Figure 8). Following the assumption that the provision of highlighting serves a focusing function, low prior knowledge students seemed to be appropriately supported by the highlighting, while high prior knowledge students were not affected either positively or negatively by the provision of highlighting. It could be assumed that high prior knowledge students are already able to connect the verbal explanation to the visual representation and therefore the highlighting might not have any additional value (Kellman & Massey, 2013; Rau, 2017). In the case of low prior knowledge, students’ attention might have been guided in a way that helped them to develop a stronger focus on relevant parts of the representation. Thus, this focusing function supports students’ visual understanding, i.e. the ability to select information from external representations, organise them into internal representations and to finally integrate them by mapping the internal representation to domain knowledge (cf. Mayer, 2014; Schnotz, 2014). This assumption is supported by the finding of a significantly higher comprehensibility, as perceived by the students in the dynamic group compared to the control group (cf. Figure 10). This pattern of results is reconcilable with multimedia learning theories (Ainsworth, 2006; Mayer, 2014; Schnotz, 2014) and with the development of representational competencies as described in chemistry education research (Daniel et al., 2018; Kozma & Russell, 2005; Rau, 2017; Stieff et al., 2011; Tippett, 2016; Treagust et al., 2003).

The interpretation of a focusing function is also supported by considerably higher average causal mediation effects in the two treatment conditions. Here, the total effect from pre to post measurement is partly mediated via the retention score. In addition, the mediation is moderated by the static and dynamic highlighting techniques, with the highest proportion mediated in the dynamic group (cf. Table 3).

Although the superiority of dynamic highlighting could not be conclusively clarified, evidence from our data suggests the added value of dynamic highlighting for students’ learning. The effects were mainly small, but substantial in the present sample and in the short duration of the intervention. However, we assume that in some areas of chemistry and other STEM disciplines there might be cases where static cues are equally beneficial as dynamic cues.

In line with previous studies investigating the effects of cues in other disciplines (Boucheix & Lowe, 2010; de Koning et al., 2009; van Gog, 2014), our study shows that the provision of highlighting compared to an un-cued instructional format had beneficial effects on fostering students’ visual understanding of representations. In addition to replicating the benefits of the signalling principle, our study provides evidence that a purposefully scripted explanation, which follows a theory-grounded structure (Caspari et al., 2018), increases students’ visual understanding independently from the type of highlighting provided.

7. Limitations

One limitation of the present study lies in the content of the tutorial videos, since nucleophilic substitution reactions are rather introductory in the field of organic chemistry. This limitation is supported by rather high pre-test scores across all groups. However, the case comparison tasks shown in the tutorial videos were a rather new task format for most learners. Therefore, we assume that the purposefully designed mechanistic explanations given in the tutorial videos supported learners in selecting and organising their prior knowledge systematically. Nonetheless, it remains to be observed if our findings are generalisable across other topics in organic chemistry as well as other domains, especially in the field of STEM, where the usage and comprehension of representations are deemed a fundamental competency (Gilbert & Treagust, 2009).

Another limitation is the short duration of the intervention. Larger learning gains are potentially possible through more videos, while longer interventions also raise questions in terms of how long this kind of support is beneficial for learning or at which point this support can be faded out. Finally, the question of the confounding effect of content (introductory topic), task format (unfamiliar/unknown) and video format (unfamiliar) arises. These effects affect all groups equally, but could above all influence the level of learning gains.

8. Conclusion and implications

Despite the aforementioned limitations, our results support the application of highlighting in instructional formats and provide evidence that a purposefully designed highlighting fosters student visual understanding. The findings call for quality assurance in instructional formats and attention guidance appropriate to the provided content. Still, it needs to be investigated how tutorial videos of this kind could be included into existing organic chemistry courses or seminars, as there is a reported need for specific instructional material (Popova & Jones, 2021). Furthermore, it remains to be systematically observed if highlighting techniques should be provided for all new contents in a lecture or if they might turn into a distractor when advanced learners already possess a certain degree of visual understanding.

Although the effect of signalling is well-documented in the literature, the actual processing and usage of visual highlighting by the learner are not yet fully understood. In order to support learners to profit from visual highlighting, it remains open to what extent visual guidance initiates a purely automatic attentional processing or a rather individual visual orientation that does not strictly follow the visual highlighting. It may as well depend on the complexity of the representation and task format given to the students, which influences students’ reliance on visual cues.

To conclude, our study contributes a step towards overcoming the representation dilemma by supporting students in connecting their partly fragmented ideas of chemical concepts to corresponding representations by implementing beneficial highlighting techniques and purposefully designed explanations into instructions. These instructional tools seem to provide adequate support to students’ visual understanding of reaction mechanisms as a specific type of complex representations, especially in case of students with low prior knowledge. While highlighting was shown to contribute to students’ visual understanding, further research is needed to better understand how students actually process this type of tutorial format in detail.

Acknowledgements

We thank all co-workers from the Graulich and Parchmann groups who assisted in data collection as well as the students who voluntarily participated in the study. A special thanks to Gyde Asmussen and Axel Langner who supported us throughout the whole project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the German Research Foundation DFG (Deutsche Forschungsgemeinschaft) under [grant number: 329801962].