Finger pointing to support learning from split-attention examples

Abstract

We investigated whether finger pointing is an effective cognitive-load self-management strategy to mitigate the split-attention effect during learning. This effect holds that learning from split-attention examples consisting of spatially separated, but mutually referring text and picture, is less effective than learning from equivalent spatially integrated sources. One-hundred-and-twenty-nine undergraduates studied a picture with accompanying text about the nephron in a between-subjects design with the factors strategy use (pointing vs. no pointing) and instructional format (split-attention vs. integrated). The split-attention effect was confirmed by results on a comprehension test and a combined measure of learning effort and test performance (i.e. instructional efficiency). However, evidence for the benefits of pointing was only found for retention performance (i.e. not for comprehension performance and cognitive load ratings) for participants who learned from the split-attention example (i.e. not for participants who learned from the integrated example). Replications are invited to examine pointing as a self-management strategy.

Keywords:

• Cognitive load theory
• self-management effect
• split-attention effect
• finger pointing

Introduction

Learning materials consisting of a combination of text and pictures are increasingly used in education because research has shown that students learn better from words and pictures than from words alone (i.e. multimedia principle, Mayer, 2014). However, presenting text and pictures simultaneously does not guarantee effective learning, such as when mutually referring text and pictures are presented at a spatial distance from each other. As an example of this presentation format, it can be seen in Figure 1(a) that learners cannot read over the text and the picture at the same time and must split their attention between both information sources. Research based on cognitive load theory (CLT; Sweller et al., 2019), a theoretical framework providing guidelines on learning and instruction based on human cognitive architecture, provides support for this in the form of the split-attention effect (Ayres & Sweller, 2022; also called the spatial contiguity effect, Mayer, 2021). This effect holds that learning from split-attention examples consisting of spatially separated, but mutually referring text and picture, is less effective than learning from examples consisting of the equivalent spatially integrated sources (for an example see Figure 1(b)). According to CLT, the searching and matching process of information in the split-attention examples imposes a high load on working memory, thereby negatively affecting mental integration processes, which are prerequisite for effective learning (Pouw et al., 2019).

Figure 1. Learning Materials Used in This Study Explaining the Structure and Function of The Nephron. (a) Split-Attention Format; (b) Integrated Format.

Within the context of CLT, one solution to optimise learning performance from split-attention examples is presenting the spatially separated information closer to each other, which can reduce the need to engage in search-and-match processes. This approach which has been the focus of CLT studies in the last three decades is characterised as the instructor management of cognitive load because the instructor changes the way the information is presented to learners (Mirza et al., 2019). However, it is unrealistic to expect all physical (e.g. in textbooks) and online (e.g. on websites) learning materials to be (re)designed based on CLT principles. Learners are exposed to a considerable number of sub-optimally designed materials consisting of text and pictures. Research in a broader field of learning and instruction has yielded valuable suggestions on how learning strategies can be used to enhance learning (Fiorella & Mayer, 2015). Different from the learning strategies that focussed on fostering generative learning (e.g. summarising, self-explanation), CLT researchers have proposed that learning strategies can also be used to manage cognitive load (e.g. Mirza et al., 2019). Learners can be equipped with self-management strategies to more effectively learn from instructional materials which have not been designed in accordance with CLT principles.

Self-Management strategies for Split-Attention examples

The first empirical investigation on the use of self-management strategies can be found in the study of Roodenrys et al. (2012). They showed that university students obtained better learning performance and experienced lower mental effort after being instructed to use a pen to highlight circle keywords, and draw arrows to link the corresponding information in the text and picture when learning from split-attention examples compared to students who were not instructed to use this strategy (for similar findings, see Sithole et al., 2017). Another self-management strategy was investigated by Tindall-Ford et al. (2015), who guided secondary school students to transform a split-attention example into an integrated example by manually dragging and dropping adaptive text segments as close as possible to the corresponding parts of the picture. Results revealed higher transfer performance for learners taught to apply the self-management strategy than those who were not taught to apply this strategy. A key characteristic that these two strategies have in common is the focus on making changes to the learning materials. However, in most situations, it is not possible or allowed to physically integrate picture and text by manipulating learning materials, such as when learning in noninteractive digital learning environments. De Koning et al. (2020a) showed that instructing learners to mentally integrate text and picture in split-attention examples can be an effective self-management approach in such environments. They gave university students specific instruction to imagine dragging and dropping text segments to the associated elements in a picture as a new self-management strategy. Students who used this mental integration strategy performed better than students who did not use the mental integration strategy with split-attention examples and performed as well as students who received the instructor-managed approach providing an integrated format. This finding broadens the application scenario of self-management to the situation where split-attention examples cannot be physically manipulated by learners. In this study, we contribute to this and investigated whether finger-pointing can be used as a self-management strategy during learning from non-manipulable split-attention examples.

Finger pointing to aid learning

Finger pointing is a well-developed type of hand movement in humans. In this study, we focussed on finger pointing (further referred to as pointing) in its role as a cognitive prompt to support learners’ own understanding and defined it as a deictic action against a surface. Deictic action means that no representational content is included in the action (Schüler & Wesslein, 2022). For example, when learners are asked to point at the text and its corresponding locations in the picture, the action itself does not contain additional information. Currently, most studies in the field of multimedia learning have been focussed on tracing, the dynamic movements of pointing. Tracing can contain both deictic and representational aspects. For example, when tracing a sentence, the action is deictic, while tracing pictures with visuospatial information can be representational because it helps depicting the visuospatial structure in mind. Several studies have investigated representational tracing, showing that students who were instructed to trace geometry worked examples achieved better learning outcomes compared to students not instructed to trace (e.g. Du & Zhang, 2019; Ginns et al., 2020). Some studies investigated how both pointing and tracing can enhance learning. In the study of Macken and Ginns (2014), the effect of pointing and tracing was examined with university students who needed to learn about the anatomy of the heart from text and pictures. Students who pointed and traced on the integrated text-picture information performed better on retention and comprehension tests than those who did not point and trace. This study was later extended by Korbach et al. (2020), in which information was presented in a split-attention format instead of an integrated format. Results showed that pointing and tracing enhanced the identification performance and moreover, led to longer fixation time at the pictures and more transitions between the text and pictures. More recently, Ginns and King (2021) replicated the effect of pointing and tracing in a computer-based lesson of astronomy. Additionally, they found that university students who pointed and traced did not only perform better on retention and transfer but also reported lower extraneous cognitive load and higher intrinsic motivation than those who did not point and trace. In general, supportive evidence for the effects of pointing and tracing on multimedia learning is increasing. However, because finger-pointing and representational tracing were combined in previous studies without considering their individual contributions, it is unknown yet how finger-pointing contributes to the beneficial effects found for tracing and pointing. Specific effects of finger pointing have only been studied in fundamental behavioural research on memory performance such as the recall of locations, which suggested that pointing could both impair source memory (e.g. Ouwehand et al., 2019) and improve it (e.g. Ouwehand et al., 2016).

Two theoretical perspectives may explain the beneficial effects of pointing on learning. First, pointing can guide visual attention meaning that faster and more accurate attention is allocated to the place where the finger points at than where no finger points at. Therefore, the to-be-learnt information is cued and becomes more salient with positive effects on learning (Schneider et al., 2018). Not only the pointed place but also the surrounding information, the perihand area, is strengthened in visual processing because of the near-hand effect (Agauas et al., 2020). Participants recognise information faster and process the information deeper when presented in the near-hand area (e.g. Reed et al., 2010). The fundamental functioning of pointing can contribute to successful learning as it aligns with the assumptions of the cognitive theory of multimedia learning (CTML; Mayer, 2021). In CTML, selecting relevant information is an important step for the organisation and integration of information. Therefore, learners who use pointing benefit from more explicit attention of the pointed area, which supports the mental integration of multiple sources of information.

Second, pointing can be performed effortlessly and offload cognitive load. According to the evolutionary account of CLT (Geary, 2008; Paas & Sweller, 2012), pointing can be regarded as biologically primary knowledge, which refers to the knowledge that people gain inherently and effortlessly. This differs from biologically secondary knowledge that people can only gain under the specific guidance and with considerable effort, such as reading. Due to the limited capacity of working memory, it is critical that working memory resources are used efficiently. This can be achieved by using biologically primary knowledge, which is less affected by working memory limitations, to assist in the acquisition of biologically secondary knowledge, which requires more working memory resources. Pointing can be cognitive offloading as it can, for example, help locate a visuospatial element effortlessly and therefore release the working memory resources previously occupied by remembering the visuospatial location needed for mental integration of associated textual and pictorial information. In other words, pointing can enhance the construction of high-quality schemas and thus benefit learning (Paas & Sweller, 2012).

Present study

The aim of this study was to investigate whether instructing learners to use a pointing strategy when learning from split-attention examples would increase learning outcomes (i.e. retention, comprehension) and reduce cognitive load (i.e. difficulty rating, mental effort rating). The contributions of the present study are twofold. First, we uniquely focussed on pointing only instead of the combination of pointing and tracing used in previous studies. Second, we included split-attention and integrated formats together in a single study, which has not been done in previous studies (e.g. Macken & Ginns, 2014). Participants were guided to either use their index finger(s) to point to the text and picture or not point at all, and they were provided with materials either in a split-attention format or in an integrated format. Participants’ working memory capacity (WMC) was also assessed as a covariate because WMC may influence the split-attention effect and the effectiveness of pointing. Learners with lower WMC have been shown to perform worse in learning from split-attention examples (Fenesi et al., 2016) and benefit more from task-related gesture using in problem-solving (Eielts et al., 2020). Specific hypotheses were as follows.

Firstly, based on CLT and research on the split-attention effect (Sweller et al., 2019), we expected a split-attention effect: when no pointing instructions are given, participants presented with an integrated format would obtain higher retention (Hypothesis 1a) and comprehension (Hypothesis 1b) and report lower difficulty (Hypothesis 1c) and mental effort (Hypothesis 1d) than those presented with a split-attention format.

Secondly, according to the evolutionarily perspective of CLT (Paas & Sweller, 2012) and empirical studies on pointing and tracing (e.g. Ginns & King, 2021), we predicted positive effects of pointing when learning from a split-attention format. Specifically, when presented with a split-attention format, participants who use a pointing strategy would obtain higher retention (Hypothesis 2a) and comprehension (Hypothesis 2b) and report lower difficulty (Hypothesis 2c) and mental effort (Hypothesis 2d) than those who do not use a pointing strategy.

Thirdly, we predicted that the positive effects of pointing would be less pronounced for integrated examples than for split-attention examples because learning from integrated examples requires less support for mental integration. Thus, participants who use a pointing strategy would obtain slightly higher or equal retention (Hypothesis 3a) and comprehension (Hypothesis 3b) and/or report slightly lower or equal difficulty (Hypothesis 3c) and mental effort (Hypothesis 3d) compared to those who do not use a pointing strategy.

Method

Participants and design

One-hundred-and-twenty-nine undergraduate students from Erasmus University Rotterdam participated. An a priori power analysis conducted using G*Power (version 3.1.9.6; Faul et al., 2007) indicated that at least 128 participants were needed to detect a small effect (f = .25) with 80% power using F-testing between means with $\alpha$ = .05. Three participants were excluded due to incompliance with the instructions. Of the remaining 126 participants (72 females and 54 males, M_age = 21.11 year, SD_age = 2.63), 88.1% were right-handed and 91.3% were non-English native speakers. Participants reported a sufficient level of English proficiency (M = 7.57, SD = 1.20) and low prior knowledge on the learning topic (M = 1.07, SD = 0.40). The experiment language was English. Ethical approval was obtained from the Ethics Review Committee at Erasmus University Rotterdam. Participants participated voluntarily and gave informed consent. A 2 (strategy use: (pointing vs. no pointing) × 2 (instructional format: split-attention vs. integrated) between-subjects design was employed. Participants were randomly allocated to one of the four experimental conditions: pointing with split-attention format (n = 32), pointing with integrated format (n = 32), no pointing with integrated format (n = 32), and no pointing with split-attention format (n = 30). All participants received course credits or 10 Euros as a reward. A monetary reward (i.e. 10, 30, 50 Euros) was given to the three best-performing participants on the subsequent tests for the learning task.

Materials

Demographics and prior knowledge questionnaire

Demographic information on age, gender, handedness, and English proficiency was collected with a Qualtrics questionnaire. English proficiency was reported by participants on a 9-point rating scale (‘1 Beginner’, ‘5 Intermediate’, ‘9 Advanced’). Participant’s prior knowledge on the learning topic (i.e. the nephron) was measured by a self-rated 5-point scale (‘1 - none at all’, ‘2 - A little’, ‘3 - A moderate amount’, ‘4 - A lot’, ‘5 - a great deal’). If a participant chose ‘2′ or higher, an open question followed asking the participant to list everything (s)he knew about this topic. If the participant’s answers were incorrect, the prior knowledge rating would be recoded as ‘1 – none at all’ as this would provide a more valid indication of the actual prior knowledge of the participants based on the more detailed information.

Working memory capacity

A standard automated operations span task (OSPAN) developed by Unsworth et al. (2005) was used to measure working memory capacity. OSPAN has been shown highly predictive of an individual’s academic performance across a range of contexts and can be regarded as a general measure of cognitive functioning. In this task, an array of letters (varying from 3–7 letters per trial) was presented at the beginning of each trial. Then, an arithmetic problem followed (e.g. (1*2) + 1 =?). Participants needed to recall all letters in the correct sequence after solving the arithmetic problem. The task consisted of 75 trials. For each letter that was recalled in the correct position in the array, 1 point was assigned. Per the participant, the minimum score was 0 and the maximum score was 75.

Learning materials

The learning materials from Cierniak et al. (2009) were adapted by translating the text from German to English and presenting the structural and functional information on the same page. Three pages were included: one small-size paper (210 × 297 millimeters) presenting the background knowledge of the learning topic (i.e. nephron, the basic unit of the kidney, see Appendix 1), one small-size paper (210 × 297 millimeters) presenting the strategies to be used by the participants in the upcoming learning task (Appendix 2), and one large-size paper (297 × 575 millimeters) presenting the content of the learning task (i.e. the structure and function of the nephron) in text and picture (in colour; Figure 1). The instructions for pointing strategies were adapted from Macken and Ginns (2014). Participants in the pointing conditions were instructed to use their index finger(s) to link related information in several ways and participants in the non-pointing conditions were asked to sit on their hands while learning the materials. These instructions explicitly asked participants to (not) use hands based on previous gesturing studies (e.g. Pouw et al., 2020). An example of a text and picture of the human eye was presented along with each instruction for participants to practice the to-be-used pointing strategies. The learning content was presented in split-attention format or integrated format.

Learning outcome measures

Learning outcomes were measured with a retention test and a comprehension test based on Cierniak et al. (2009). The retention test assessed how well participants remembered the structure of a nephron. Participants were asked to select the correct terms from a list of 14 structures to match the 10 blanks shown in the picture. One point was given for a correctly matched structure with a maximum of 10 points. Cronbach’s $\alpha$ of the retention test was 0.74. The comprehension test contained 12 multiple-choice questions, which assessed how well participants remember and understand the functioning of the nephron. These multiple-choice questions were designed in a way that mental integration of the text and picture was necessary to answer them correctly. An example question is: Which one of the following structures carries blood that has been filtered away from the glomerulus? Four alternatives of A. Renal artery, B. Afferent arteriole, C. Efferent arteriole, and D. Proximal convoluted tubules were indicated by the arrow in the picture. Four alternatives were provided and only one was correct. For each question, 1 point was given for a correct answer. A total of 12 points on the comprehension test could be attained. The Cronbach’s $\alpha$ of the comprehension test was 0.45.

Cognitive load measures

The perceived cognitive load was measured by two 9-point subjective rating scales developed by Paas (1992). Participants were asked to rate the perceived amount of invested mental effort from ‘1 - very, very low mental effort’ to ‘9 - very, very high mental effort’ and task difficulty from ‘1 - very, very easy’ to ‘9 - very, very difficult’ after the learning phase and after each test. These scales have been extensively used in CLT research (Sweller et al., 2019) and are assumed to provide a valid and reliable measure of overall cognitive load (for an overview see Paas et al., 2003; Ouwehand et al., 2021).

Procedure

Participants were tested individually in a four-person lab at [XXX]. The lab was equipped with four desktop computers that were separated by dividers and a maximum of four participants could be tested per session. First, the OSPAN task was administered. Then, participants’ demographic information and prior knowledge were collected via Qualtrics. Next, participants engaged in the learning task, for which the experimenter handed out the paper-based materials in order. Participants first had two minutes to read the introduction to the kidney and the nephron. Then they were given the instruction page and had three minutes to familiarise themselves with the strategies of the appropriate use or no use of their hands and practiced with an example. Subsequently, participants studied the text and picture to understand the structure and functions of a nephron to the best of their abilities within 12 min. During the study phase, each participants’ learning behaviour (i.e. hand movements) was recorded by a camera placed in front of them. After learning, participants completed the cognitive load ratings for the learning task. Next, participants completed the retention test (max. 4 min) and comprehension test (max. 6 min), after each of which they completed the cognitive load ratings. The experiment lasted approximately 60 min.

Data analysis

Compliance with the instructions was determined by evaluating the time spent on pointing in each condition, which was derived from the videos recorded in the learning phase. For each participant, the total time spent on pointing was calculated. If participants pointed for at least 1 second, they were considered non-compliant if they were in the no-pointing conditions and compliant if they were in one of the pointing conditions. The evaluation was done by two independent raters using ELAN 6.0 (The Language Archive, 2020). The two raters coded approximately 20% of videos together and showed high interrater agreement (r = .99) and then they independently coded half of the remaining videos respectively. Three participants (one in the no pointing with split-attention condition, no pointing with the integrated condition, and pointing with the split-attention condition, respectively) were excluded from the analyses due to incompliance with the instructions. The rest of the participants complied with the instructions (pointing with the integrated condition: M = 10.50 min, SD = 0.38, pointing with split-attention condition: M = 10.81 min, SD = 0.32, no pointing with the integrated condition: M = 0.00 min, SD = 0.00, no pointing with split-attention conditions: M = 0.00 min, SD = 0.00).

All data were analysed in SPSS 27 and R. In SPSS, separate one-way analyses of variance (ANOVAs) were conducted to compare the four conditions on prior knowledge, English proficiency, and OSPAN score. Two-way ANCOVAs with OSPAN score as the covariate were conducted to test the main effects of instructional format and strategy use and their interaction on learning outcomes and cognitive load measures. Planned contrasts using Sidak correction were conducted in R to test the hypothesised pointing effect and split-attention effect. One-tailed p-values were reported for planned contrasts.

Results

There were no significant differences between conditions in prior knowledge, F(3, 122) = 2.05, p = .111, η_p² = .05, and English proficiency, F(3, 122) = .77, p = .511, η_p² = .02. Also, no significant differences were found between the experimental conditions on the OSPAN score, F(3, 122) = .73, p = .535, η_p² = .02. Table 1 shows means and SDs for learner prerequisites, learning outcomes, cognitive load ratings.

Table 1. Means (SDs) for English proficiency, prior knowledge rating, OSPAN score, learning outcomes, cognitive load ratings, and instructional efficiency.

		Pointing with integrated format condition	Pointing with split-attention format condition	No pointing with integrated format condition	No pointing with split-attention format condition
		M (SD)	M (SD)	M (SD)	M (SD)
English proficiency		7.75 (0.19)	7.34 (0.22)	7.50 (0.22)	7.70 (0.24)
Prior knowledge rating		1.00 (0.00)	1.06 (0.06)	1.19 (0.11)	1.00 (0.00)
OSPAN score		37.94 (2.69)	39.88 (3.34)	43.53 (2.98)	42.47 (2.80)
Learning outcomes	Retention test	6.31 (2.73)	6.94 (2.02)	6.66 (2.43)	5.37 (2.68)
	Comprehension test	5.72 (2.43)	4.84 (1.89)	6.09 (2.10)	5.30 (2.17)
Task difficulty	Learning task	7.38 (1.07)	6.59 (1.43)	6.69 (1.60)	6.93 (1.20)
	Retention test	6.28 (1.71)	5.78 (1.16)	5.71 (1.63)	6.53 (1.57)
	Comprehension test	7.88 (0.87)	7.68 (1.14)	7.38 (1.16)	7.90 (0.76)
Mental effort	Learning task	6.47 (1.29)	6.28 (1.46)	5.84 (1.50)	6.23 (1.30)
	Retention test	6.28 (1.37)	5.94 (1.50)	5.69 (1.51)	5.87 (1.28)
	Comprehension test	6.16 (1.72)	6.58 (1.26)	6.13 (1.54)	6.40 (1.67)
Instructional efficiency	Retention test	−0.14 (0.97)	0.13 (0.84)	0.27 (0.94)	−0.29 (0.83)
	Comprehension test	−0.06 (0.90)	−0.25 (0.89)	0.38 (0.89)	−0.08 (0.81)

Learning outcomes

Retention test

No significant main effects were found of instructional format, F(1, 121) = .58, p = .448, η_p² = .01, and strategy, F(1, 121) = 2.18, p = .143, η_p² = .02. There was a significant instructional format × strategy interaction, F(1, 121) = 4.52, p = .036, η_p² = .04. Planned contrasts showed that the difference between the no pointing with the integrated condition and the no pointing with split-attention condition was not significant, t(121) = 2.03, p = .064, d = 0.52, thus Hypothesis 1a was not confirmed. However, the pointing with split-attention condition outperformed the no pointing with split-attention condition, t(121) = 2.53, p = .019, d = 0.63, which confirmed Hypothesis 2a. No significant difference was found between the pointing with the integrated condition and no pointing with the integrated condition, t(121) = −0.45, p = .479, d = −0.34, so Hypothesis 3a was confirmed.

Comprehension test

There was a significant main effect of instructional format, F(1, 121) = 4.74, p = .031, η_p² = .04. The integrated conditions (M = 5.91, SD = 2.26) outperformed the split-attention conditions (M = 5.07, SD = 2.00), which confirmed Hypothesis 1b. No significant main effect of the strategy was found, F(1, 121) = 1.15, p = .287, η_p² = .01, and there was no significant interaction, F(1, 121) = 0.01, p = .914, η_p² = .00, thus Hypothesis 2b was not confirmed and Hypothesis 3b was confirmed.

Cognitive load measures

Difficulty rating

No significant main effect of instructional format was found on the difficulty rating for the learning task, F(1, 121) = 1.26, p = .264, η_p² = .01, for the retention test, F(1, 121) = .32, p = .575, η_p² = .00, or for the comprehension test, F(1, 120) = .80, p = .373, η_p² = .01. Similarly, no significant main effect of strategy was found on the difficulty rating for the learning task, F(1, 121) = .61, p = .435, η_p² = .01, for the retention test, F(1, 121) = .04, p = .834, η_p² = .00, or for the comprehension test, F(1, 120) = .96, p = .328, η_p² = .01. However, there was a significant interaction effect on the difficulty rating for the learning task, F(1, 121) = 4.67, p = .032, η_p² = .04, for the retention test, F(1, 121) = 6.04, p = .015, η_p² = .05, and for the comprehension test, F(1, 120) = 4.50, p = .036, η_p² = .04. However, planned contrasts showed no further significant differences on the difficulty ratings (p > .05), except that participants in the pointing with integrated condition reported significantly higher difficulty for the comprehension test than participants in the no pointing with integrated condition, t(120) = 2.21, p = .042, indicating no support for Hypotheses 1c, 2c; only Hypothesis 3c was partially confirmed.

Mental effort rating

No significant main effect of instructional format was found on the mental effort rating for the learning task, F(1, 121) = .15, p = .700, η_p² = .00, for the retention test, F(1, 121) = .11, p = .738, η_p² = .00, or for the comprehension test, F(1, 120) = 1.54, p = .218, η_p² = .01. Similarly, no significant main effect of strategy was found on the mental effort rating for the learning task, F(1, 121) = 2.46, p = .119, η_p² = .02, for the retention test, F(1,1 21) = 1.97, p = .163, η_p² = .02, or for the comprehension test, F(1, 120) = .20, p = .655, η_p² = .00. In addition, no interaction effect between instructional format and strategy was found on the mental effort rating for the learning task, F(1, 121) = 1.56, p = .215, η_p² = .01, for the retention test, F(1, 121) = 1.14, p = .288, η_p² = .01, or for the comprehension test, F(1, 120) = .06, p = .813, η_p² = .00. Thus, Hypotheses 1d and 2d were not confirmed and Hypothesis 3d was confirmed.

Exploratory analysis on instructional efficiency

Visual inspection of the means of the learning outcomes and mental effort ratings suggested that some instructional formats may have been more instructionally efficient than others. According to Paas and van Merrienboer (1993; Paas et al., 2003) learners’ behaviour in a particular instructional condition is more efficient if their performance is higher than might be expected on the basis of their invested mental effort if their invested mental effort is lower than might be expected on the basis of their performance, or both. For example, the data presented in Table 1 suggest that learning in the no pointing with integrated format condition required the lowest mental effort investment, but resulted in the highest comprehension test performance. To explore the possible effects of instructional format and strategy on instructional efficiency, we used the adapted formula developed by Paas and van Merrienboer (1993: see also Van Gog & Paas, 2008), which used a standardisation procedure to combine the data on mental effort during learning with performance data on the test. This measure of instructional efficiency has been adopted in many studies to provide an additional efficiency view on the instruction (for overviews, see Paas et al., 2003; Van Gog & Paas, 2008). Separate instructional efficiency scores were calculated based for retention test and comprehension test performance. Table 1 shows means and SDs for instructional efficiency scores.

Retention test

For the instructional efficiency score of the retention test no significant main effects of instructional format, F(1, 121) = 0.84, p = .362, η_p² = .01, and strategy, F(1, 121) = 0.01, p = .936, η_p² = .00 were found. However, there was a significant instructional format × strategy interaction, F(1, 121) = 7.25, p = .008, η_p² = .06. Planned contrasts showed that the no pointing with split-attention condition had a lower instructional efficiency than the no pointing with the integrated condition, t(121) = 2.53, p = .0.02, d = 0.652, which is consistent with the split-attention effect. No significant differences were found between the pointing with split-attention condition and no pointing with split-attention condition, t(121) = 1.83, p = .010, d = 0.42, and between the pointing with the integrated condition and no pointing integrated condition, t(121) = −1.97, p = .074, d = −0.48.

Comprehension test

For the instructional efficiency score of the comprehension test a significant main effect of instructional format was found, F(1, 121) = 4.25, p = .041, η_p² = .03. The split-attention conditions (M = −0.17, SD = 0.85) showed lower efficiency than the integrated conditions (M = 0.16, SD = 0.92). There was also a significant main effect of strategy, F(1, 121) = 4.50, p = .036, η_p² = .04, on the instructional efficiency score of the comprehension test. The pointing conditions (M = −0.15, SD = 0.89) showed lower efficiency than the no-pointing conditions (M = 0.16, SD = 0.88). No significant instructional format × strategy interaction was found, F(1, 121) = 0.84, p = .362, η_p² = .01.

Discussion

This study extends the self-management of cognitive load research by examining whether pointing can be used as an effective self-management strategy to enhance learning from split-attention examples. For this, strategy use (pointing vs. no pointing) and instructional format (split-attention vs. integrated) were manipulated and learning outcomes as well as a cognitive load during the learning and testing phase were assessed.

We first hypothesised that a split-attention effect would be obtained with the current learning materials, which was confirmed for comprehension test performance (Hypothesis 1b) but not for retention test performance (Hypothesis 1a) and cognitive load ratings (Hypotheses 1c/1d). These findings are partly consistent with those found by Cierniak et al. (2009), who found a split-attention effect on both learning outcomes and subjective cognitive load ratings using the same materials. Some aspects might have contributed to the discrepancy in the findings. For example, we made minor adjustments to the learning materials and tests to suit our research aim (e.g. presenting the structure and function information on the same page instead of two separate pages). We also used a different sample and language in the materials (English) which was not the native language for most of our participants while Cierniak et al.’s materials and participants were German. Moreover, we used single-item cognitive load scales rather than scales that can distinguish different types of cognitive load. Despite these differences, an explorative analysis of instructional efficiency revealed the split-attention effect on the retention and comprehension tests, indicating that the split-attention conditions led to significantly lower instructional efficiency than the integrated conditions. Therefore, our findings provide a basis for testing whether pointing is an effective strategy to support learning from split-attention examples.

The finding that the pointing effect was found with the split-attention format for retention test performance (Hypothesis 2a), but not comprehension test performance (Hypothesis 2b) suggests that the effect is restricted to remembering information and does not seem to contribute to a better comprehension of the information. Corroborating evidence for this comes from the study of Korbach et al. (2020), in which the effect of pointing and tracing was only found on the less complex and more visually oriented identification test but not on the more complex and less visually oriented comprehension test. Some explanations can be discussed for these findings. First, the benefits of pointing may be more salient in less difficult/complex tasks. We used a complex task because instructional principles generated by CLT are only expected with complex tasks imposing a high cognitive load. We assumed that pointing could help reduce the cognitive load of the mental integration process by its visual-attention-guiding function and its effortless offloading function. However, these functions may also apply to simple tasks. Some studies on pointing using simple memory tasks showed that pointed objects were better remembered in young and old adults (e.g. Ouwehand et al., 2016). Pointing at objects, in addition to simply seeing them, can enrich the constructed schema and supply alternative avenues for recalling the memory later on. Therefore, the benefits of pointing can be reflected in the retention test because it relies more on the visuospatial elements (Paas & Sweller, 2012). However, for complex tasks such as comprehension of information where deeper cognitive processing is required, pointing may contribute little on this part. In other words, pointing may benefit learning not only by reducing the load. The underlying mechanism is unknown yet and further investigation is required to uncover this. Future research could examine the pointing effect using less complex learning materials.

Second, how we instructed students to point can also have contributed to its (lack of) effectiveness. Our instructions may not fully guarantee active engagement in cognitive processing. Students were required to point on the related text and picture elements, but without explicit explanation on why this should be done, which could have resulted in a passive way of implementing the pointing strategy. So, students might have been behaviourally active while being less engaged cognitively. Previous studies have indicated that ‘how and why’ information is both important for self-management instructions to support learning (e.g. De Koning et al., 2020a, 2020b). Therefore, it is important to increase awareness of why the pointing strategy should be used, which could be addressed in future studies.

For students who learned from an integrated format, pointing was not effective for both retention and comprehension test performance, which was in line with Hypothesis 3a and 3b. This finding suggests that pointing may become less useful when the need for visual attention guidance is lower, as was the case in learning from an integrated format. This contrasts with studies that investigated the combined effects of pointing and tracing on learning from integrated materials regarding (e.g. Ginns & King, 2021; Macken & Ginns, 2014). Whereas the benefits of pointing and tracing on retention, comprehension, and transfer were found in these studies, we failed to find comparable effectiveness when only pointing was used. One explanation could be that the beneficial effects of pointing and tracing are connected to the representational nature of tracing. Studies including tracing normally used materials containing either process information (i.e. directional arrows, steps) or specific graphical representations in learning materials (e.g. Du & Zhang, 2019). Therefore, the differential effectiveness of pointing and tracing could be influenced by the number of representational elements contained in the instructional materials. In future research, the individual and combined effects of pointing and tracing could be investigated. This could also help to make more definite claims about the effectiveness of pointing in integrated materials.

Regarding cognitive load, no support was found for reduced cognitive load ratings with pointing in the split-attention format condition (Hypothesis 2c and 2d). There was support for Hypothesis 3c (partially) and Hypothesis 3d predicting no difference on cognitive ratings between the pointing and no pointing condition with the integrated format. Moreover, participants in the pointing with integrated format condition even reported higher difficulty for the comprehension test than those in the no pointing with integrated format condition. The instructional efficiency score for the comprehension test also showed that the pointing conditions had lower efficiency than the no-pointing conditions. Possibly pointing may simultaneously decrease the cognitive load associated with searching and matching processes and increase the cognitive load due to conducting the pointing action in the learning process, while the latter may hinder learning. Similar explanations were proposed by Korbach et al. (2020) and Rossi-Arnaud et al. (2017). For the integrated format, the positive aspect of pointing was not needed, but the negative aspect of pointing materialised and led to a higher difficulty rating. Future studies could focus on both the positive and negative effects of pointing which could help improve our understanding of the effects of pointing in split-attention and integrated materials. Nonetheless, it should be noted that the findings regarding cognitive load are consistent with those found in other studies into the split-attention effect (Schroeder & Cenkci, 2020) and the self-management effect (e.g. De Koning et al., 2020a), in which single-item mental effort ratings also failed to distinguish the total cognitive load between conditions. Future studies could consider using scales that specify different types of cognitive load (e.g. Leppink et al., 2013) and/or physiological measures such as eye-tracking. In addition, the cognitive load and performance results suggest that solely considering cognitive load or performance measures may be less informative than looking at the combined measures in terms of instructional efficiency.

Limitations and Future directions

Our study had several limitations. The instruction for the no-pointing conditions, which required students to sit on their hands to keep their hands still, might have interfered with the measures taken. Future studies could use a more ecologically valid control condition such as giving general instructions on ‘do not use your hands while you learn the materials’ or not providing specific instructions on where to place their hands (Ginns et al., 2020). We also do not know whether participants in the no-pointing conditions fully used the time for learning, which could possibly have influenced the results. Additionally, the relatively low reliability of the comprehension test, which could be due to the multi-faceted nature of the questions, could have influenced the results. Participants were given limited learning time, which could have reduced the possibility to develop the level of understanding needed to answer the (comprehension) questions as well. For future research, it would be helpful to develop tests with higher reliability, have longer learning time, or both. Moreover, although participants indicated sufficient English proficiency, presenting the materials in a non-native language could have caused unnecessary cognitive load and affect the results. Future studies should try to avoid such a potential confounding influence.

A few more directions can be considered for future studies. First, the design of the split-attention task could be considered in investigating the effects of pointing. In our study, the labelling of text and picture that was used could already have provided sufficient attentional guidance to match the spatially separated textual and pictorial information. In this case, the pointing strategy could be redundant, and therefore less effective. Future studies could compare the pointing effect in conditions with different levels of attentional support. Second, we only used the WMC score as a covariate and did not investigate the extent to which differences in WMC (e.g. high vs. low) were related to the effects of pointing. Future research could further explore whether the stronger effect of hand movements on learning observed for people with lower WMC in research on problem-solving (Eielts et al., 2020) would also apply for self-management strategies using hand movements like pointing. Lastly, the motivational effects of pointing in addition to the cognitive effects could be investigated. According to CLT, pointing as biologically primary knowledge can be intrinsically motivating (Paas & Sweller, 2012). Evidence has shown that including hand movements (e.g. tracing) in the lesson can increase students’ intrinsic motivation and learning performance (Wang et al., 2022). Therefore, investigating the motivational aspect of pointing can expand its generalisation and provide more insights on its beneficial effects.

Conclusion

The present study provides partial evidence for the possibility to instruct learners to self-manage their cognitive load by pointing when learning from split-attention examples. By using pointing to make connections between spatially separated text and picture, participants obtained a higher retention test score than those who did not point. However, this beneficial effect was not found for comprehension test scores and cognitive load ratings. Results also showed no effects of pointing for participants studying an integrated format. It is clear that it is too early to generalise these findings as further investigations are needed to examine the robustness of the pointing strategy in learning from split-attention examples.

Author contributions

Shirong Zhang: Conceptualisation, Data curation, Investigation, Formal analysis, Funding acquisition, Methodology, Project administration, Writing – original draft, Writing – review and editing

Bjorn de Koning: Conceptualisation, Data curation, Methodology, Writing – review and editing, Supervision

Fred Paas: Conceptualisation, Data curation, Methodology, Writing – review and editing, Supervision

Disclosure statement

The authors declare that they have no conflict of interest.

Data availability statement

The materials and the datasets generated and/or analysed for the current study are available from the corresponding author upon request.

Additional information

Funding

This research was supported by the scholarship from the China Scholarship Council [201706360140].

 

Appendix

1. Background Information Page

2. Instruction Page

a. Pointing With Integrated Format Condition

b. Pointing With Split-Attention Format Condition

c. No Pointing With Integrated Format Condition

d. No Pointing With Split-Attention Format Condition