Pilot Evaluation of the Use of Video Snippets to Support Undergraduate Learners

Abstract

This paper evaluates the use of video snippets to support undergraduates learning critical path analysis (a project management technique) in an intermediate level (second year undergraduate) module. The video snippets are made available in a blog that encourages students to evaluate their performance on the task before and after watching the video snippets; the students are also asked to indicate how confident they feel about performing the task at different stages in the exercise. The design, delivery and evaluation the use of video snippets is discussed, and feedback from participants at each of the three stages of deployment is considered. There were twelve participants in the study, four at each stage. The study was carried out at London Metropolitan University.

Keywords:

• learning styles
• video snippets

1. Perception of Need

Students often need support in addition to timetabled lectures, seminars, workshops and academic staff office hours; asynchronous feedback to supplement face-to-face activities would be particularly useful, as timetabled sessions are not necessarily at the time when help is required by the students. An email or telephone call is not always sufficient as no demonstration is possible.

Although some research (as expressed by Greenfield (2007) cited by Henry (2007)) has suggested that learning styles and learning cycles are not an accurate reflection of the way students learn, this author believes that such techniques can be employed to serve as useful indicators, rather than be taken as prescriptive approaches in the analysis, design and evaluation of learning materials.

Taking as an example of a learning cycle (Kolb (1984) cited in Davies and Lowe (n.d.)), it can be seen that different styles of learning fit around it; Honey and Mumford ((1982), cited in Atherton (2011)) in Figure 1 below, indicate that different support would be required by learners employing different learning styles, and at different points in their learning journeys.

2. Learning Styles

Various theories and models about the way people learn have been proposed. These include a spectrum of learning styles and a range of learning cycles. Honey and Mumford (cited in Atherton (op cit) and Fewings (n.d.)) propose Activist, Pragmatist, Theorist and Reflector as learning styles.

Figure 1 Learning Cycle with Learning Styles (Atherton (2011))

Felder and Silverman (1988, cited by MindTools) contrast scales of learning styles: sensory to intuitive; verbal to visual; active to reflective; and sequential to global.

Memletic styles identified in Advanogy.com (2004) specify visual, logical, aural, verbal and physical, and introduce the solitary versus social aspects, where a learner might tend to think things through on their own or prefer learning as a group activity. Further learning styles identified and displayed in the BBC — Open University website (n.d.) include: auditory, kinaesthetic, visual, socio-emotional and metacognitive.

There is some overlap between these classifications of learning styles. A diagram devised by this author (Figure 2 below) shows how different approaches to learning can be envisaged as a collection of styles which can be integrated by individuals when learning new things.

Figure 2 Integrating learning styles

The extent to which each learning style is employed by an individual could be represented by a three-dimensional version of the jigsaw, so that a style that is used more is taller than another which is used less. In this way, the jigsaw will appear like a contour map of applicable learning styles for an individual. The same method could also be used for illustrating the learning styles supported by a particular teaching and learning technique, to show how well the learning styles are supported by a particular approach. Another method would be to code the jigsaw pieces by colour to indicate the degree to which they are employed (red [hot] for a highly used style, yellow [warm] for a moderately used style, blue/cyan [cold] for a style only marginally used). An alternative colour scheme would be traffic lights red-yellow-green with red the least used style and green the most used, but care would need to be taken so it is clear which colour scheme is employed to avoid confusion.

This collection of learning styles is not exhaustive, and their use is intended as guidance rather than being definitive. The same learner studying different topics might use different learning styles, for example, learning to ride a bicycle (a physical activity) is likely to require a different learning strategy from learning to cook (where sensory and reflective approaches might yield more beneficial results) or from synthesising a logical proof (more theorist, logical and sequential in nature). The learning styles of individuals may vary over time, as they “learn how to learn”. It is useful to have a taxonomy of learning styles in order to prepare materials for learners, so that provision can be made for different approaches and the omission of certain styles is avoided.

3. Method

The method selected for implementing this study was action research. This approach employs a cycle of activity: formulating ideas, testing, gathering feedback, reflecting and amending. In this study the cycle has run three times, with feedback being gathered at each stage and used to refine and enhance the material for subsequent stages.

4. Existing Work in the Area

An email to Multimedia colleagues in the Faculty of Computing elicited two responses from staff who use video in their teaching, but these did not relate to critical path analysis.

Other work researched includes discussion of the use of video technology in teaching, including examples outside the subject area under consideration, for example teacher training videos (Stannard, 2006 — 2011).

5. Analysis of Existing Provision (videos on critical path analysis)

Investigation revealed YouTube videos related to critical path analysis, and the suitability of existing video snippets evaluated and classified in Figure 3 below.

Figure 3 Categorisation of existing video material for critical path analysis

As can be seen from Figure 3 above, some of these videos use a different notation [e.g. SirGanttalot (2008a and b) nptelhrd (2010)], some are presented as a voice-over on a still image GreenfordMaths (2009), whereas others show the presenter working through examples on blackboard nptelhrd (2010) or whiteboard s3lokchand (2008). One example was a manual demonstration, but without written or spoken explanation nkistaff (2010). None was in talking head format; either there was no person visible, or most of an individual was shown (in which case, the person’s body often obstructed the view of the diagram). None of the videos encouraged reflection on the part of the learner.

These existing tools are not considered suitable for use within the intervention for the following reasons:

• presenter unknown to students — lack of rapport;

• different notation employed — possibility of confusion;

• do not offer learners the opportunity to try an example for themselves;

• do not ask learners to reflect on their experience and evaluate their own skills;

• do not incorporate or encourage learner reflection;

• speaker moving in front of whiteboard/blackboard can be distracting;

• do not invite feedback from the learners;

• some involve advertising (distracting and inappropriate).

It was therefore decided that some video snippets would be recorded specifically for use within this study.

6. The Intervention

6.1. The Technology Employed

A series of video snippets was made to demonstrate the technique of critical path analysis to undergraduate learners. This task involves simple arithmetic calculations in order to arrive at a value for the duration of a project, and also identifying areas where any delay would cause the project to overrun. The video snippets are embedded in a blog post for viewing by the target users. An initial video snippet provides an introduction and an overview of the task as a whole, and the remaining three of the video snippets include a narration recorded while the task is being performed. The narrations were recorded in a single take, and subsequently subtitles were added (as a result of feedback from stage 1 of the study).

6.2. Target Module and Audience

The module selected for the intervention was CC2005 Project Planning and Management, an intermediate level (second year undergraduate) module delivered in the spring semester which is core on the B.Sc. Business Information Technology (and available on other degrees). These learners require additional practice in order to gain confidence and expertise in using the technique.

Over a period of about ten years, it has been noted that some students have difficulty in the use of critical path analysis. Some feel intimidated by the visual impact of the diagram itself; other learners are uncertain of the appropriate choice of value as a result of performing calculations.

The purpose of the use of this technology is to provide students with personal individual on-demand access to an explanation and demonstration of a key tool in project management (that of calculating the critical path in a network diagram) by means of a video demonstrating the technique, with a description of what is happening at each stage in the process. This allows learners to review key points in a way that they would not be able to do in a lecture or seminar context.

As the target learners have already met the topic, the video assists in reinforcing and building on existing knowledge, rather than introducing new material. It is intended that the use of a task, then the videos, followed by further task, will assist learning in that it echoes Kolb’s (op. cit.) learning cycle.

7. Approach

One talking head video was made as an introduction, so that students would see a familiar face, and have an overview of the material to be covered. Students could choose to watch all three videos, or select between them. Three demonstration video snippets showing a printed network diagram manually annotated with explanatory narration, firstly calculating earliest event times (EETs), secondly calculating latest event times (LETs) and finally identifying the critical path. The same diagram was used in all three video snippets so that learners could see how each stage built on the previous one. The video snippets were posted on a purpose-made blog at: http://how-to-cpm.blogspot.com/ together with instructions on how they should be used (including encouragement for learner reflection and self-evaluation) and a request for learner feedback.

The videos created by the author are now categorised using the same criteria as employed for analysing existing videos available on YouTube for critical path analysis as shown in Figure 3 above. Note that the author’s videos are shorter than most of the existing material, and are also embedded in a blog which encourages reflection and participation. The subtitles were prepared and uploaded based on the words spoken, rather than using the YouTube “Transcribe Audio — Beta” voice recognition facility which sometimes selects inappropriate words (e.g. “aromatics” in place of “arrow method”).

Figure 4 Categorisation of the author’s video snippets for critical path analysis used in this study

Figure 4 above illustrates the results of classifying the video snippets; the introductory video employs the talking head approach, while the remaining three use a voice-over accompanying a paper-based step-by-step manual demonstration of the critical path technique.

Jonasson et al (1999) state:

“Video is an additional technological tool the authors advocate to support constructivist learning. Under the traditional paradigm, film and videos are merely shown to students in a passive manner. Constructivist learning employs video as an active tool that requires learners to produce information, as opposed to consume information. Learners must be active, constructive, intentional, and cooperative to produce video.”

In this intervention learners are able to put what has been gleaned from the video snippets into practice. The students are not passive observers; they are initially asked to carry out the task of identifying the critical path and to state how confident they feel in performing this task (self-reflection). The next step is for the students to watch the video snippets; they may watch and review the video snippets as many times as they feel necessary (guidance of allowing 20 – 30 minutes for the task as a whole). Students are then asked to reflect on their level of confidence after watching the video snippets. There is then a further similar exercise for students to undertake, and they are asked to evaluate their own performance once more. This means that multiple learning styles are employed and supported.

8. Evaluation of Intervention

The intervention was performed in three stages as shown below. As there were only four participants at each stage, the results are not statistically significant, but they are indicative that a further study involving a lager number of participants would be worthwhile.

Stage 1: Initial feedback from three colleagues and a past graduate

Individuals with prior knowledge of the technique were consulted in order to detect any issues with the material that needed corrective action. Ten colleagues were approached, and feedback was received from three of these, and a past graduate who is familiar with the technique.

Feedback from colleague 1: “This looks really good. The worked examples are very clear and good sound quality is good too. The intro video looks good but has a few problems with the dialogue as it is out of sync. Did you optimise it for web streaming? I really do feel this is the way forward fro many modules as it gives the students the chance not only to review the lecture material, but also to be taken through it. The only trouble is that it does take a long time to do and get it right.”

Feedback from colleague 2: “Looks really good. Can I let my Masters students at it? I just explained Critical Path to them last week very very briefly! I do have a comment — is it possible to produce a transcript of what you say in the introduction, so that the user can see what you are saying in addition to listening?”

Feedback from colleague 3: “You can consider using sub titling as well as voice and this also helps students with written language - accuracy, spelling and vocabulary.- as also good for hard of hearing students.”

Feedback from past graduate: “Video 1, good, but not very good sound quality. Video 2 / 3 / 4, Very good, sound quality fine. You may want to add subtitles for accessibility purposes.”

As a result of this feedback, subtitles were provided on each of the video snippets by transcribing what had been recorded in the voice-over in each video snippet.

Stage 2: Trial with four final year students, with questionnaire and discussion

Feedback was obtained from four final year students as participants in the second stage of the study. Although initially it was planned that this would be on a one-to-one basis, this was carried out with a group of four students and supplemented by individual discussion; it was therefore possible to see how they interacted as a group. This interaction demonstrated that the technique of using video snippets also supports the socio-emotional learning style.

Subject A had not attempted critical path analysis before, but said he was confident in undertaking the task. When asked about this, he explained that he was confident that he would be able to carry out the task as a result of watching the videos, although he recognised that he could not perform the task straight away (i.e. he was not confident that he would be able to carry out the task prior to receiving support from the video snippets). This subject attempted the task after watching the video snippets, and although he had made mistakes, was able to recognise errors he had made, which demonstrates that the video snippets had enhanced his understanding of the task. Further development of the questionnaire to elicit learner feedback for stage 3 was made as a result of feedback from this learner. Subject A received comments from subject B after they had watched the video snippets, and was content with the progress that he had made. It is worth noting that this subject was using the video snippets as an ab initio learning tool, rather than for clarifying or reinforcing existing knowledge, and that he had some success with this approach.

Subject B offered guidance to Subjects A and C after watching the video snippets, explaining to them what their mistakes were as a result of watching the video snippets. He was able to perform the task correctly after watching the video snippets, but had made a minor arithmetic error before viewing them.

Subject C stated that he felt more confident in performing the task as a result of watching the video snippets, and was able to correct minor errors in his calculations. He engaged with subject B in discussing the results of the task, and stated that he felt confident in his ability to attempt such a task in the future.

Subject D found “it all came back” as he had not used the technique for some time. This subject had used the technique two years previously (when he took the module for which the current students are the target audience; he had subsequently spent a year on an industrial placement); he had not used the technique in the intervening period. This subject found that the video snippets reassured him that his understanding and use of the technique was correct, and reminded him of each step in the process.

Stage 3: Release of material for target audience, with online questionnaire

The video snippets were made available on purpose-built blog incorporating the questionnaire, with access details in the module blog and WebLearn (the virtual learning environment in use at London Metropolitan University).

The following feedback from learners was received during this stage of the study:

“It is really helpful for the students who are working on their module project as well as for the students who working on their degree dissertation” Stage 3, Learner 1

“I really found it very useful, thank you for your time making this video to help us. I wish the university support your ideas of making tutorial video and special blog for this module, teachers should start from next [academic year] to create blogs and to upload some tutorial videos to help the students to review the lectures again at home” Stage 3, Learner 2

“I’m with <Learner 2> :D” Stage 3, Learner 3

“The videos were great, they helped me understand what critical path was and how to calculate the EET and the LET. Thank you. They helped me understand what seemed quite daunting. When I first saw the diagrams I thought that I would have to put on my A Level Maths cap on to work out the critical path (a cap I haven’t worn for a long time). And the videos explained what needed to be done in a simple yet thorough way. Thank you.” Stage 3, Learner 4

It was possible to ascertain the use made of the video snippets by means of the statistics available in the blog where the video snippets and questionnaire were available (Figure 5 below). Feedback was obtained from four students in the target audience; two students posted positive comments on the blog, one emailed answers to the questions (this subject found increased confidence in the activity after watching video snippets); two students added themselves as followers of the blog; one student posted feedback in the early stages of the study.

Figure 5 Daily accesses to video snippets used in this study

The video snippets developed for this study are able to satisfy the characteristics of most of the learning styles highlighted in Figure 2 above as shown in Figure 6 below:

Figure 6 Evaluation of learning styles supported by video snippets in this study

Red shows that visual, verbal, sensory, intuitive, aural (and associated learning styles) are well supported.

The learning styles kinaesthetic, pragmatist and active are shown as yellow (moderate) as it is recognised that students might watch the video snippets without trying out the activities for themselves. Similarly, while the intervention encourages learners to reflect on their own skills, it cannot be certain that this is taking place when the video snippets are being watched asynchronously (although this did take place for stage 2 of the study).

The solitary and socio-emotional learning styles are also shown as moderate because the video snippets will support either, but it will not be possible to tell which is in use as learners view the snippets in their own time. In stage 2 the group of four students interacted and helped each other after watching the video (more than they did with the task before watching the video).

The global learning style is shown in cyan as this is not heavily involved (except, perhaps, in the introduction video snippet which gives an overview).

9. Further Work

It would be possible to identify understandable mistakes and explain why they are in error; care will need to be taken to avoid information overload with this approach.

It is not intended to incorporate this intervention into formal assessment to avoid pressure this might invoke and allow it to be used as and when required (rather than as an exercise in collecting marks without making real use of the material).

Further work can be undertaken to improve lip sync issues in the talking head video; this is not an issue for the demonstration video snippets as there is no image of the speaker.

10. Analysis

Each video snippet was recorded in one take, leading to a smoother and more natural delivery than following a script. Subtitles were added by transcribing and adding the text, rather than speaking to a script; this wastherefore a more natural delivery of material. Feedback was obtained from each of the three stages of the study (four participants providing feedback in each stage) enabling reflection and adaptation of the material.

Not all stage 3 participants responded directly to the online questions about the intervention, as gauged by the number of daily accesses, i.e. there is evidence of more use made of the video snippets than is demonstrated by the feedback from users. Feedback from learners at this stage was requested via the video snippet blog (as a blog comment or by email), and the use of material was monitored via the statistics tab in the blog as shown in Figure 5 above.

It was found that in stages 2 and 3 of the intervention that learners gained confidence in the technique, were able to view and review the technique in their own time and able to identify their own mistakes. Learners used the video snippets as a review and revision aid, as can be seen from Figure 5, where there were more accesses to the material on the three days before the examination and the day of the examination itself than at other times.

The learning materials produced for this study can be accessed in university PC rooms (including the library, where the subtitles are of benefit to users without headphones), from students’ own PCs, and also via smartphones, allowing access in a variety of locations and at times that suit the learners.

11. Conclusion

Video snippets are well-received by students as a supplement to existing methods of delivery. It is worth investigating other areas which would lend themselves to the technique; the author’s plan is to create video snippets to supplement lectures on normalising data for databases (a technique which many students often find difficult) in second year undergraduate BSc and FdSc modules on business database applications which run in the autumn semester. Further studies with greater number of participants would enable a study to be undertaken to establish whether there is an effect of statistical significance on the use of video snippets in the areas of student engagement, performance, satisfaction and confidence in using techniques demonstrated in this way.