Making educational videos more engaging and enjoyable for all ages: an exploratory study on the influence of embedded questions

ABSTRACT

There is Individual variation in how people interact with videos presented in online distance education. Educational videos can be embedded with interactive content to increase engagement and make cognition more efficient. Accordingly, we predicted that embedding questions during videos (rather than after) would enhance the performance of question-answering and be preferred by students. We also hypothesised that the benefits of presenting questions during videos might increase with age. Using a counter-balanced within-subject design, each participant watched short videos with questions embedded either during the video or presented after the video, and we then surveyed their experiences. Although there were no differences in correct responses, participants answered questions posed during videos more efficiently than questions presented after. Females enjoyed questions during videos more than males. Younger individuals (e.g. 25–34) seemed to benefit more from questions during videos than slightly older students (35–44). Interestingly, with increasing age (from 25 to 74), there was a shift in preference towards answering questions after, rather than during, videos. Overall, embedding questions was an effective and well-liked method for enhancing the interactivity of module-related videos. The age of students should be considered when embedding questions.

KEYWORDS:

• Distance education
• lifelong learning
• interactive videos

1. Introduction

Educational videos are often vital resources for supporting instruction in virtual learning environments (VLEs). The use of VLE videos has increased during the COVID-19 pandemic, as many education providers have adopted distance learning to deliver tuition (Dedeilia et al., 2020; Iwanaga et al., 2021; Pokhrel & Chhetri, 2021; World Health Organization, 2021). Importantly, distance learners are a heterogeneous group, with a higher representation of older, working, and female individuals than non-distance learners (Chen et al., 2019; Latanich et al., 2001). Therefore, to improve the development of VLE-based educational videos, we need to understand individual variations in the benefits of video interactivity and people’s preferences; we take an experimental approach to investigate such issues.

While many VLE videos are recorded lectures (Cummins et al., 2016; Lavigne & Risko, 2018; Phillips et al., 2016; Szpunar et al., 2014; van der Meij & Bӧckmann, 2021), videos can also present virtual experiential learning opportunities, including lab or patient demonstrations, case-study interviews, and descriptions of structures or mechanisms that are difficult to verbalise in traditional lectures (Arguel & Jamet, 2009; Kestin & Miller, 2022; Mayer & Chandler, 2001; Mayer et al., 2003; Torres et al., 2022). According to the Cognitive Theory of Multimedia Learning, such narrative videos can help students ‘build a mental model of a cause-and-effect system’ that reduces cognitive load and subsequently improves their learning (Mayer & Chandler, 2001, p. 396). While this is helpful when guiding learners to acquire knowledge (Noetel et al., 2021), it is critical not to overload students with complex information presented in videos. To further reduce cognitive load, videos can be segmented into smaller chunks, and signals can be used to help focus a student’s attention (Ibrahim, 2012). Accordingly, the present experiments explore how to improve learning efficiency associated with VLE-based demonstration videos by dividing them into sections with embedded questions. We investigated performance on Question Embedded Videos (QEVs) and students’ preferences for including QEVs in online modules. We were particularly interested in how individuals vary in their experience with QEVs, and thus we assessed differences across gender identities and age ranges.

The use of ‘pop-up’ questions within narrative demonstration videos aligns well with ‘flipped classroom’ examples of education (Chouhan, 2021; Haagsman et al., 2020; Lo & Hew, 2017; O’flaherty & Phillips, 2015) and is supported by constructivist models of learning (Vural, 2013). Accordingly, engagement with the narrative videos is self-paced, and students are forced to discover information independently; instructors don’t deliver lessons but are instead mediators of knowledge. Students can connect video content with their previous experiences; this personalisation of understanding can help reduce cognitive load (Torres et al., 2022) and may be especially useful for experience-rich adult learners (Khalil & Elkhider, 2016). Cognitive load may be reduced because students employ a viewing strategy, such as connecting with experiences or finding ways to optimally answer questions (Costley et al., 2020).

The reduction in cognitive load afforded by QEVs might help motivate students to engage with module materials and improve student satisfaction (Haagsman et al., 2020; van der Meij & Bӧckmann, 2021; Vural, 2013). Furthermore, using test-like questions, as opposed to other forms of interactivity, within videos likely enhances student performance on later exams (Cummins et al., 2016; Kestin & Miller, 2022; Szpunar et al., 2013, 2014; Torres et al., 2022; Yang & Xie, 2021). It has long been known that guiding students through videos with paper-based questions can improve student module performance (Lawson et al., 2006). Taking tests (e.g. learning to retain and retrieve knowledge during videos with questions) may promote test-taking ability on later assessments (Jacoby et al., 2010). Alternatively, there may be an indirect relationship between improved module performance and earlier experience with QEVs; such videos may improve focus, encourage note-taking, and enhance the amount of time spent studying (Haagsman et al., 2020; Lawson et al., 2006; Szpunar et al., 2013; Vural, 2013). This manuscript examines how the placement of questions within narrative (non-lecture) videos impacts performance and satisfaction.

Most research on the efficacy and experience of QEVs in classes studies typical undergraduate-age students (e.g. around 21 years of age; Cummins et al., 2016; Haagsman et al., 2020; Torres et al., 2022; van der Meij & Bӧckmann, 2021). Lifelong learning can have many benefits on the health and well-being of individuals, with advantages ranging from decreasing depression and enhancing social satisfaction (Laal & Salamati, 2012; Narushima, 2008; Weinstein, 2004). Older students may prefer to learn via video (e.g. ages>41; Simonds & Brock, 2014), possibly because they learn at their own pace and desire access to audio-visual materials (Heaggans, 2012; Weinstein, 2004). Such preferences may be related to the increased cognitive effort required for tasks as we age (Hess & Ennis, 2012), and videos may reduce this effort. Our study hypothesises that by reducing cognitive load (Cummins et al., 2016; Torres et al., 2022), QEVs may help older participants learn and enhance their viewing experience.

Adults may be primarily motivated to study because an employment requirement for a university degree is becoming the norm. Therefore, additional education may be required for individuals who have decided to re-enter the workforce (Osam et al., 2017). Despite this, adult education can be challenging to deliver, as student retention levels may need to be improved when individuals need to balance work/life commitments with their coursework (Renner & Skursha, 2022). Therefore, education for adults should be governed by principles of humanism, whereby individuals have control over their personal growth and are responsible for their independent learning (Arghode et al., 2017). More specifically, andragogy is a humanistic approach to studying adult learning, theorising that prior experiences contribute significantly to adult learning and that adults are highly self-directed in their education (Arghode et al., 2017; Khalil & Elkhider, 2016). Accordingly, QEVs may be beneficial andragogic approaches, as they promote asynchronous, flexible and self-paced learning. However, focussing on techniques associated with andragogy may be inadequate, as they often overlook how learning strategies previously used by the individual may contribute to (or hinder) their adult learning, and thus may not fully acknowledge individual variation in preferred learning methods (Arghode et al., 2017).

Studies of ‘flipped classroom’ or online-distance education often do not detect differences in performance between people identifying as male or female (Chen et al., 2019; Yu, 2021). Despite this, females may be more engaged in their education, while males may have a more stable positive perception of online learning (Nistor, 2013; Richardson & Woodley, 2003). While there is significant individual variation, it has been suggested that enhanced female engagement and confidence during online learning, relative to males, may be related to their ability to multi-task and resist distractions (Alghamdi et al., 2020; Price, 2006; Stoet et al., 2013). In part, enhanced engagement with online education for females may be facilitated by interactions with tutors (Price, 2006) and the desire to access supplemental online resources (possibly including videos; Martin & Bolliger, 2018). Thus, while we anticipate not detecting differences in question-answering performance between males and females, we may observe that females show an enhanced preference for engagement opportunities, such as QEVs, in online education.

The present work investigates whether splitting online videos into multiple interactive segments impacts question-answering performance (e.g. correctness, question-answering speed), as well as increases engagement and satisfaction. Accordingly, we inserted questions into existing online videos, measured performance on these prompts, and identified how students felt about the experience. Notably, the videos presented to participants were already produced, and questions were inserted using commercially available software. Therefore, if adopted for teaching, this approach requires minimal effort from the instructor, primarily requiring the creation of formative or summative test questions to be inserted into existing videos. Accordingly, we aimed to develop a replicable strategy for improving learning from videos and, at a minimum, to create a more engaging experience for asynchronous learning (particularly for distance learners). Crucially, we analysed data across age ranges (25–74 years old) and gender identities; such detailed analyses can provide a more nuanced view of how the delivery of video-based educational material can be optimised to individual needs.

2. Materials & methods

2.1. Participants

Participants (n = 158; ranging from 18–74 years of age) were selected from the Curriculum Design Study Panel (CDSP) at The Open University. We compared question performance and survey responses between individuals aged 25–34 (n = 14), 35–44 (n = 17), 45–54 (n = 35), 55–64 (n = 33), and 65–74 (n = 15); these are a more comprehensive range of ages compared to other studies, highlighting a novel aspect of our dataset (Cummins et al., 2016; Haagsman et al., 2020; Torres et al., 2022; van der Meij & Bӧckmann, 2021). We excluded 44 cases because they were incomplete or contained extreme outliers, leaving 114 (74 identified as female, 40 identified as male) complete cases for analysis. Importantly, it remains problematic that most pedagogical literature focuses on comparing people who identify as male or female (Paechter et al., 2021); no participants reported identifying as non-binary in our random sample.

This study panel is voluntary, comprises current students at The Open University, and is refreshed annually. Consent is gained through self-selection for the study panel. The experiments and questions were approved by the University Human Research Ethics Committee (The Open University) and by the CDSP. Participants were randomly assigned to different groups; therefore, we can assume equal variation in background knowledge and experience between the designated groups. The experiment was completed by the participants remotely in their own homes. The study was anonymous; therefore, participants could not contact one another for answers to questions. A within-subjects design was used, and each participant watched three videos. These videos had questions embedded during playback or questions placed after the video. Each participant experienced the same set of questions, regardless of their location.

2.2. Videos and embedded questions

Questions were specific to video content, so internet searches could not easily be used to find answers. The videos and embedded questions were already used as text-based online module material at The Open University. Individuals were excluded from participating if they were currently or previously enrolled in the modules from which the videos were taken. Since the videos were specially produced for the modules, students would not have previously viewed them.

All experiments were conducted via The Open University VLE and PlayPosit, a video platform that allows adding interactive features to standard video and provides analytics. The video files were hosted in PlayPosit, ensuring consistent question delivery across the different groups and experiments (Supplement A, Figure S1). PlayPosit served as a ‘wrapper’ for the videos, allowing the questions to be seamlessly embedded during or after the video. As shown in the figure, participants answered questions by clicking a radial button and hitting ‘submit’ (participants were familiar with this procedure via Open University modules and did not require training). Immediate feedback was provided (green highlights for correct answers and red for incorrect).

Three videos on blindness, malaria, and first responders, were used in all experiments and were part of the University curriculum. The chosen videos had to meet four criteria: (1) The videos had to be already in presentation and used in current modules at The Open University to increase the experiment’s external validity and avoid the added workload of creating new novel resources. (2) The videos selected should not have been previously viewed by the selected cohort of participants. (3) The selected videos had to be understandable by a lay audience. (4) The videos had to differ in duration. While each video was less than 5 minutes long, viewing the videos while answering questions lasted approximately 20–30 minutes (completed in one sitting).

Questions were based on the retention of the information presented in the videos, and each video had three embedded questions. An example question would be ‘Which of the following is a symptom of malaria?’, with potential multiple-choice answers including ‘Low body temperature’, ‘Severe headaches’, ‘Constipation’, and ‘Excessive energy’ (see Supplement B for a complete list of questions). For QEVs presented during videos, questions occurred approximately 1 minute after giving relevant information. We did not create the multiple-choice questions for the videos; questions were designed according to Open University guidance and were already displayed adjacent to the associated videos (e.g. click to unhide an answer) on module websites. After finishing the QEVs, participants completed a questionnaire and were allowed to engage with an online feedback forum.

2.3. Survey & forum

After the QEVs, participants were given a survey (another 20 minutes). The survey was delivered using the Questionnaire tool of The Open University Moodle VLE (see Supplement C for questions). Briefly, survey questions first recorded demographics and previous experience with online learning and video viewing habits during modules (including whether they had previously encountered QEVs; most hadn’t, so previous QEV experience wasn’t factored into the analysis). We then assessed how they interacted with the QEVs we presented, including whether they re-watched portions of the video (i.e. used the ‘rewind’ function), took notes, enabled captions, or changed the speed of the video. We did not control these features because we wished to assess more ‘natural’ student behaviour (i.e. how they would typically behave if they encountered videos in modules). However, we found that there was primarily individual variation in notetaking, so this was the only watching-related behaviour we assessed. Finally, we measured participant opinions and satisfaction with the QEV experience. We evaluated, for example, whether participants enjoyed answering questions in videos and how motivated they were to perform well on the task. We also asked participants whether they preferred questions located during or after videos. Finally, CDSP participants could interact on a forum after completing the study to provide comments on the QEVs. We did not have enough comments on this forum to conduct a detailed qualitative analysis (this was not the study’s primary purpose); instead, we highlighted a few remarks in the results section of this manuscript.

2.4. Data analysis

Data were normalised using log transformation to analyse QEV performance and time to answer. These dependent variables were assessed using two-way ANOVAs (separate for males and females), with responses to questions posed during or after videos as the within factor and age group as the between-factor. Where appropriate, post-hoc Šídák multiple comparison tests followed the ANOVAs.

Wilcoxon Signed-Rank tests were used to analyse survey-based data to compare within-subject opinions on questions asked during versus after videos. Responses to individual survey questions were analysed using Mann-Whitney U tests (males vs females) or Kruskal-Wallis H tests (across age groups, followed by Dunn multiple comparison post hoc tests).

3. Results

3.1. Descriptive statistics and previous experience

Because all participants were students at The Open University (a distance-learning institution), everyone had reported previous experiences watching non-lecture videos to support module content. Students typically watched every video available during modules (56.14% watched all videos, 25.44% most, 4.39% half, and 14.04% few). Most students (84.21%) watched each module video once, whereas only 3.51% reported watching most videos more than once. While students could download module videos for offline viewing, most individuals preferred only to watch videos online (50.88%), and a minority of participants liked watching videos offline (4.39%). Interestingly, many students reported variation in their behaviour, with a substantial number watching module videos both online or offline (21.05%), having no preference (20.18%), or responding that their viewing behaviour ‘depends’ on the situation (2.63%). Critically relevant to our present study, most participants had never before experienced QEVs in their coursework or elsewhere (82.20%).

We needed to assess whether students downloaded module videos for offline viewing; if students preferred watching videos offline, they might not have access to interactive content within videos in our VLE. In our survey, we received some written responses (18) regarding why individuals might choose to download video content. Most of these responses concerned students planning where they would study and whether they would have reliable internet access to watch videos. Participants explained that they were more likely to download videos for later viewing if their internet access was poor or if they were travelling. Thus, when teaching, it is essential to consider whether students can access online QEVs and if poor internet access might disadvantage some students if QEVs are an examinable component of classwork.

3.2. Question-answering performance

We were interested in whether individual participants performed differently on questions asked during versus after videos. Our within-subject analysis found no significant difference in the average scores for questions posed during or after videos. There were also no differences between age groups and gender identity in scores. Perhaps unsurprisingly, participants who reported taking notes (n = 23) during the videos scored better than those who did not (n = 91). This effect was significant for answering questions during videos (U = 713.5, p = 0.005; Scores for note-takers 97.8 ± 1.58% vs non-note-takers, 87.6 ± 1.81%) and displayed a strong trend for questions asked after videos (U = 806.5, p = 0.057; Scores for note-takers 94.2 ± 2.25%, vs non-note-takers 85.8 ± 2.02%). Because of these high scores, most participants appeared to retain the information presented in the videos. Therefore, we were interested in whether patterns of video engagement (e.g. speed of answering) differed across individuals and according to where questions were placed (i.e. during or after the video).

Participants answered questions posed during videos significantly faster than those presented after watching (Figure 1; Females, F(1,69) = 38.34, p < 0.0001; Males, F(1,35) = 26.59, p < 0.0001; Separate 2-way ANOVAs for males and females, with age as the between factor and question position as the within factor). Post-hoc Šídák multiple comparison tests found significant differences in question-answering speed (during vs after) for ages 25–34 (p = 0.0069), 45–54 (p = 0.0002), and 55–64 (p = 0.0001) in females, and for ages 35–44 (p = 0.017), 45–54 (p = 0.030), and 55–64 (p = 0.010) in males. Thus, for both males and females, 65–74 year-olds were the only age range where question-answering speed did not differ based on question location (during or after videos). Finally, we did not detect differences between males and females in question-answering speed for questions positioned during or after videos. Note-taking did not significantly impact question-answering time.

Figure 1. Speed of answering questions: both females (a; n= 74) and males (b; n=40) answered questions faster when they were positioned during videos than after videos (****,p<0.0001). The speed of answering questions was normalised vis log transformation. Bars represent age ranges (±SEM) and individual data points (grey circles) are shown.

3.3. Opinions on QEVs

Next, we surveyed participants’ opinions on the experience of answering questions posed during or after videos. Participants responded favourably to answering questions about video content, regardless of when questions were presented (Table 1). Our within-subject analysis found that participants enjoyed answering questions during videos more than after (Wilcoxon, Z = −2.21, p < 0.026). This was supported by qualitative feedback we received on QEVs, including that they ‘liked and felt comfortable with the format’, ‘thought the design was brilliant and tailored to the user’s experience’, believed that ‘answering the questions helped to reinforce what I had learned’, and that thought that ‘giving questions intermittently does help in learning and assimilating information’.

Table 1. Overall comparison of attitudes to questions asked during vs after videos (ages and gender identities combined). Results show averages (out of 10; ±SEM) and Wilcoxon test statistics (Z & p). n = 114; *, p < 0.05.

Question	Mean Score (SEM) for Questions During (D) or After (A) Videos	Statistics
‘I enjoyed having to answer questions that were related to video content’.	D: 7.86 (0.23) A: 7.44 (0.21)	Z=−2.21 p=0.026*
I would have preferred not to answer any questions related to video content’.	D: 2.93 (0.26) A: 2.96 (0.26)	Z=−0.14 p=0.89
‘I concentrated on correctly answering questions that were related to video content’.	D: 8.49 (0.20) A: 8.31 (0.20)	Z=−1.78 p=0.075
‘I felt that answering the questions helped me to understand the video content’	D: 7.45 (0.26) A: 7.42 (0.24)	Z=−0.44 p=0.66
‘I wanted to achieve high marks for the questions I was asked’	D: 8.66 (0.21) A: 8.60 (0.21)	Z=−1.63 p=0.10
‘I am highly motivated to watch a video if questions are posed [during or after]’	D: 7.56 (0.25) A: 7.52 (0.24)	Z=−0.15 p=0.88

According to separate analyses across gender identifications, females enjoyed answering questions during videos more than males (Table 2 and Figure 2; U = 1117.5, p = 0.027; Females, 8.19/10 ± 0.27; Males, 7.25/10 ± 0.41). Compared to males, females also thought that questions presented during videos helped them to understand video content (U = 1082, p = 0.015; Females, 7.89/10 ± 0.30; Males, 6.63/10 ± 0.47). There were no other differences in opinions between males and females regarding questions posed during or after videos. Interestingly, these results aligned with our finding that, when taking classes, females tended to engage with videos more than males by re-watching videos (U = 1163, p = 0.007; although, both males and females are equally likely to watch most videos at least once, U = 1449, p = 0.84; there were no significant differences in how students watched videos during modules across age ranges).

Figure 2. Comparison of attitudes to questions asked during vs after videos between males (n = 40) and females (n = 74). Statistics are presented in Table 2. Survey responses are scored out 10. Females are shown in white bars and males in black (±SEM). Grey circles represent individual data points. *, p < 0.05.

Table 2. Comparison of attitudes to questions asked during vs after videos between males (n = 40) and females (n = 74). Results are also presented in Figure 2. Mann-Whitney test statistics are shown in this table (U & p). *, p < 0.05.

Question	During Video Experience (males vs females)	After Video Experience (males vs females)
‘I enjoyed having to answer questions that were related to video content’.	U = 1117.5, p = 0.027*	U = 1387.5, p = 0.58
‘I would have preferred not to answer any questions related to video content’.	U = 1198.5, p = 0.070	U = 1372.5, p = 0.49
‘I concentrated on correctly answering questions that were related to video content’.	U = 1234, p = 0.12	U = 1366.5, p = 0.48
‘I felt that answering the questions helped me to understand the video content’	U = 1082, p = 0.015*	U = 1265, p = 0.19
‘I wanted to achieve high marks for the questions I was asked’	U = 1287.5, p = 0.20	U = 1330.5, p = 0.33
‘I am highly motivated to watch a video if questions are posed [during or after]’	U = 1183.5, p = 0.071	U = 1334, p = 0.38

We also compared survey response regarding the experience of questions during or after videos across age ranges [25–34 (n = 14), 35–44 (n = 17), 45–54 (n = 35), 55–64 (n = 33), and 65–74 (n = 15)]. As summarised in Figure 3, across age ranges, individuals were generally positive in their opinions about QEVs. Based on post-hoc tests (Table 3), participants in the 25–34 and 45–54 age ranges were often most positive about questions posed during videos, whereas those in the 35–44 were likely the least positive. Differences in opinions across ages on questions posed after videos were less clear, but followed a similar pattern of 35–44 year-olds being least enthusiastic. Notably, 65–74 year-olds related to questions posed after videos, which may relate to their overall preference of question position (see below).

Figure 3. Comparison of attitudes to questions asked during vs after videos across ages [25–34 (n=14), 35–44 (n=17), 45–54 (n=35), 55–64 (n=33), and 65–74 (n=15)]. Statistics are presented in Table 3. Survey responses are scored out 10. Bars represent average scores (±SEM). Grey circles represent individual data points. *, p<0.05.

Table 3. Comparison of attitudes to questions asked during vs after videos across age groups [25–34 (n = 14), 35–44 (n = 17), 45–54 (n = 35), 55–64 (n = 33), and 65–74 (n = 15)]. Results are also presented in Figure 3. Kruskal-Wallis H tests with Dunn multiple-comparison tests (for specific age ranges) are shown in this table. *p < 0.0001–0.05.

Question	During Video Experience (across ages)	After Video Experience (across ages)s
‘I enjoyed having to answer questions that were related to video content’.	H(4) = 7.63, p = 0.11	H(4) = 11.6, p = 0.021* 35–44 v 65–74, p = 0.0076
‘I would have preferred not to answer any questions related to video content’.	H(4) = 14.7, p = 0.005* 25–34 v 55–64, p = 0.0050	H(4) = 9.42, p = 0.052
‘I concentrated on correctly answering questions that were related to video content’.	H(4) = 18.9, p < 0.001* 25–34 v 35–44, p = 0.019 35–44 v 45–54, p = 0.0011	H(4) = 13.9, p = 0.008* 35–44 v 45–54, p = 0.0039
‘I felt that answering the questions helped me to understand the video content’	H(4) = 12.4, p = 0.014* 25–34 v 35–44, p = 0.024 25–34 v 55–64, p = 0.047	H(4) = 9.68, p = 0.046*
‘I wanted to achieve high marks for the questions I was asked’	H(4) = 16.5, p = 0.002* 25–34 v 35–44, p = 0.023 35–44 v 45–54, p = 0.0056	H(4) = 13.8, p = 0.008* 25–34 v 35–44, p = 0.024 35–44 v 45–54, p = 0.020
‘I am highly motivated to watch a video if questions are posed [during or after]’	H(4) = 2.48, p = 0.65	H(4) = 0.453, p = 0.98

Finally, on a single scale, we asked participants to rate whether they preferred questions posed during videos (scored 0/10) to after videos (scored 10/10). There were no differences in preference between males and females (Figure 4, left; Mann-Whitney, U = 1199, p = 0.11). In contrast, there was a significant effect of age on preference (Kruskal-Wallis H Test, H(4) = 11.01, p = 0.027), with older participants seeming to prefer questions after videos than during videos (although post-hoc tests were not significant). Such a finding for older participants aligns with their higher enjoyment of questions posed after the videos (Table 3 and Figure 3).

Figure 4. Preference for questions posed during (score of 0/10) or after (score of 10/10) videos. Data compare either males vs females (a) or across ages (b). Bars represent average scores (±SEM). Grey circles represent individual data points. n=114. *, p<0.05.

4. Discussion

In online distance learners, we examined individual variation in the performance and self-reported benefits of answering questions embedded within educational videos. Participants watched three videos, and questions on video content were asked during or after the videos. While the percentage of correct answers did not differ according to question location, participants were generally quicker at answering questions posed during videos than after. Females enjoyed answering questions posed during videos more than males and felt that answering such questions helped their understanding. Despite this, males and females did not differ in their overall preference for questions being located during or after videos. Younger participants (aged 25–34) felt the most optimistic about the questions posed during the videos. Interestingly, unlike younger participants, older students (65–74) did not differ in the speed at which they answered questions posed during or after videos. In addition, 65–74 year-olds enjoyed answering questions after videos more than other groups, and they generally preferred answering questions after videos than during. Together, while the results illustrate that participants were overwhelmingly positive about QEVs, there were some differences in their experience across genders and ages; these individual differences should be accounted for when developing QEVs for specific student populations.

This study builds upon the emerging literature on how narrative demonstration videos may be valuable additions to an individual’s online education (Chouhan, 2021; Haagsman et al., 2020; Lo & Hew, 2017; O’flaherty & Phillips, 2015). Similar to previous research, we found that students overall had a positive experience with QEVs (Haagsman et al., 2020). We initially hypothesised that embedding questions during videos might uniformly improve the learning experience, as dividing videos into manageable sections may optimise an individual’s cognitive load (Mayer & Chandler, 2001). Although we did not perform psychological tests of cognitive function, we did find that the youngest group of individuals (25–34-year-olds) were often found to enjoy and benefit from questions embedded during videos more than other age groups (especially compared to slightly older, 35–44-year-olds). This contrasts with our expectations that, with increasing age, individuals might find questions embedded during videos more helpful because the videos could help reduce cognitive effort (Laal & Salamati, 2012; Narushima, 2008; Simonds & Brock, 2014; Weinstein, 2004) and promote self-paced adult learning (according to principles of andragogy; Arghode et al., 2017). It is unclear why the oldest group of students (65–74 year-olds) seemed to prefer questions asked after videos. Still, perhaps it’s related to age-related changes in selective attention and distractibility (Guerreiro et al., 2010). In our study, it is possible that questions positioned during videos disrupted concentration for specific individuals, and regaining attention was challenging once the video re-started. Since the videos we presented were relatively short (less than 5 minutes each), perhaps answering three questions within this period was too disruptive. Future research can investigate the impact of longer videos and compare results on video performance to measures of cognitive load and selective attention (e.g. a Stroop Task; Davidson et al., 2003). It may also be possible to promote attention/engagement by supplementing the embedded questions with other enhanced visuals; research suggests that combining such features is effective for teaching scientific concepts (although it hasn’t been investigated across age ranges and gender identities; Kestin & Miller, 2022).

Several other limitations and considerations for future research should be highlighted. First, while we did not fully control for computer literacy, all participants were enrolled at a distance-learning university and were comfortable with online videos and VLEs. Therefore, individuals are likely to be more efficient at using a computer than members of the general population, especially older students (Laal & Salamati, 2012; Narushima, 2008; Weinstein, 2004). That said, most participants (82.20%) had no prior experience with QEVs, supporting the generalisability of the results. Second, we did not compare genders at varying ages, as our sample size would be too small. Third, while we used a bespoke questionnaire to assess participant satisfaction with QEVs, subsequent studies may expand the survey to evaluate multiple dimensions for each response category. Fourth, our questions tested content retention and required a multiple-choice response. Future research will investigate more ‘formative’ questions that are unmarked and support creative thinking about the video content (Weurlander et al., 2012; Yorke, 2003). Such questions, for example, may encourage short-answer responses, discussions, or assess prior knowledge; students generally find such questions rewarding and motivational (Schroeder & Dorn, 2016; van der Meij & Bӧckmann, 2021; Yang & Xie, 2021). Finally, future research could integrate QEVs into live modules students take; assessments could then determine if QEVs promote long-term knowledge retention. Integrating QEVs into classwork could help determine whether the benefit of QEVs is due to a direct effect of testing (e.g. enhancing retention on specific topics) or an indirect impact of testing (e.g. increasing engagement with online resources). Similar to Haagsman et al. (2020), our current results suggest that QEVs may have an indirect impact on student performance, as students who voluntarily took notes achieved higher scores on the questions.

Together, our study supports using QEVs in teaching, particularly when used with narrative videos. Although more research is required, placing questions during videos for younger students (e.g. 25–34-year-olds) and after videos for older pupils (e.g. 65–74 year-olds) may be beneficial. Importantly, we re-used videos already produced and inserted questions into them using commercially available software; therefore, we highlight that teacher workload need not drastically increase to create QEVs. Thus, QEVs may be effective and engaging learning resources across ages, and instructors should monitor their use to ensure that individual needs are addressed and learning objectives are achieved.

Data availability

Anonymised data are available upon request by contacting the corresponding author. Surveys used can be found in the Supplemental Materials.

Ethics

The experiments and questions were approved by the Open University Human Research Ethics Committee (HREC/3239/Singer) and by the Curriculum Design Study Panel (CDSP).

Acknowledgments

We would like to thank eSTEeM, the Open University centre pedagogy in the Faculty of Science, Technology, Engineering and Mathematics centre for their support of this project. We would also like to thank Sue Germer from PlayPosit, who allowed us to engage with PlayPosit for this experiment and provided guidance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/02601370.2023.2196449

Additional information

Funding

The work was supported by the eSTEeM at The Open University.

Notes on contributors

Morgan B. Zolkwer

Morgan B. Zolkwer graduated from the University of Sussex in June 2022 (BSc Hons Psychology). He worked as a placement student and completed his undergraduate dissertation research with Dr Bryan Singer. Morgan starts his Psychology PhD at University College London in October 2022.

Rafael Hidalgo

Rafael Hidalgo holds an MSc in Electronic Engineering and a Media MBA. He joined the Open University (OU) in 2004 as a Media Project Manager and is currently a Senior Learning Designer. Before joining the OU, Rafael had a 15 years long career in the broadcasting industry, in technical and senior management roles. Rafael is currently researching the use of learning analytics as a tool to inform and improve the design of OU modules. He is also working on the use of interactive video in education, particularly in a distance learning environment.

Bryan F. Singer

Dr Bryan F. Singer (PhD, FHEA) is a Lecturer in the School of Psychology at The University of Sussex. He is also the Director of the Sussex Addiction Research & Intervention Centre and the Chair of the Cross School Research Ethics Committee at The University of Sussex. Dr Singer is also an Associate Lecturer at The Open University. Dr Singer’s teaching focuses on the biological psychology of mental health, as well as the science of memory. His research investigates the neural underpinnings of learning, motivation, and addictions. Dr Singer received his PhD from The University of Chicago and a postgraduate certificate in higher education from The University of Sussex.