Students’ Attitudes to and Usage of Academic Feedback Provided Via Audio Files

Abstract

This study explores students’ attitudes to the provision of formative feedback on academic work using audio files together with the ways in which students implement such feedback within their learning.

Fifteen students received audio file feedback on written work and were subsequently interviewed regarding their utilisation of that feedback within their learning. The written work provided by twelve of the students was also conventionally marked and this feedback was quantitatively compared with the audio feedback comments following classification of individual comments.

The students responded very positively to the audio file feedback judging it to be good quality because it was easier to understand, had more depth and was more personal. Most students annotated their work as they listened and stated that they would use the audio feedback to improve their work for other tutors. Pitfalls of this form of feedback included the large size of the audio files (up to 11Mb) being incompatible with some e-mail systems. Comment classification showed a greater emphasis on developmental aspects of learning within audio file feedback.

Guidance is provided for tutors concerning the effective use of audio file feedback. As feedback is integral to all learning this project has wide applicability across the disciplines.

Keywords:

• feedback
• mp3
• audio
• formative assessment
• learning

Introduction

Learning may concern within-person changes which modify the way in which we interpret and act towards our surroundings (Edwards, 2005). Socio-culture models have been used to make sense of how we learn, often involving learning as being constructed through the use of tools such as language to advance concepts (Vygotsky, 1978). When applying constructivist models of learning tutors commonly need to intervene through providing a kind of scaffolding process that enable the student to achieve a goal which is beyond their unassisted efforts (Wood et al., 1976). Within higher education a socio-culture model of learning exists between students and their peers and between students and tutors, and within this dynamic model of learning the language of feedback (scaffolding) enables students to achieve goals to a greater extent than they would without peers or tutors. That feedback does enhance learning has been shown through a number of meta-analysis studies (Hattie and Timperley, 2007; Black and Wiliam, 1998). Gibbs and Simpson (2004) consider feedback has the most powerful single influence on student learning and achievement.

Sadler (1989, p120) states:

’Few physical, intellectual or social skills can be acquired satisfactorily simply through being told about them. Most require practice in a supportive environment which incorporates feedback loops.’

However, for feedback to be effective it needs to be detailed, understood and used by the student to self-assess their learning (Gibbs and Simpson, 2004; Nicol and Macfarlane-Dick, 2006).

Despite its importance most academics find that providing feedback to students is time consuming (Carless et al., 2006) and, despite the time invested in it, the feedback provided may not be effectively used by all students (Higgins et al., 2002). A possible reason for this is that students need to develop their ability to use feedback (Sadler, 1989).

Sadler (1989) agues that to learn from feedback students need a) to have a concept of the reference level they are aiming for, b) an ability to compare their current standard of work with that reference level and c) to be able to engage in appropriate behaviour that enables closure of the gap between the reference level and their current level.

It is known that the form in which students receive information may influence their ability to assimilate it (Flemming, 2007). Learning styles which have preferences for visual, aural, written or kinesthetic information have been identified and, since there is evidence that students are undertaking less reading for pleasure (Martin, 2007), it is possible that students may be becoming less comfortable in processing written information such that alternative forms of feedback delivery may be more effective. Our previous study (Orsmond et al., 2005) indicated that the majority of students preferred verbal feedback from tutors since this encouraged questioning and discussion. Other research evidence (Shriver, 1991; Shriver, 1992) has shown that hearing think-aloud reading can help writers to develop a greater appreciation of the needs of readers of their work, and that this appreciation can lead to improvements in writing performance. Advances in technology mean that it is now possible for tutors to easily record and distribute spoken feedback to students via e-mail as audio files. Furthermore, the popularity of portable audio players and the plethora of multimedia programs opens up new possibilities for students to interact with such feedback.

Other studies using different electronic means of delivering student feedback have indicated that these may enhance the quality of that feedback (Pitt and Gunn, 2004) and further attractions of the use of electronic forms of feedback are that, compared to handwritten comments, it may be more easily archived, copied and distributed for quality assurance purposes. Electronic feedback, in addition to meeting many of the requirements of Gibbs and Simpson (2004) and Nicol and Macfarlane-Dick (2006) is analogous to the think aloud reading investigated by Shriver (Shriver, 1991; Shriver, 1992).

This qualitative (interview) and quantitative (feedback classification) study aims to consider the effectiveness and feasibility of providing feedback on academic work to students using audio files. Effectiveness will be considered in terms of student perceptions of audio file feedback (i.e. concerning how students use it for clarification) and also students’ implementation of such feedback (i.e. concerning how students use it in their learning, motivation and reflection). Clarification, learning, motivation and reflection were identified as the four primary uses of feedback by students in a previous study (Orsmond et al. 2005).

The findings from this study have been used to develop guidance for tutors regarding approaches to providing the most effective feedback to students in this format and this guidance is presented at the end of this paper. As feedback is integral to all students’ learning this project has wide applicability across the disciplines

Specifically, the study addresses the following questions:

• How do students respond to, understand and interpret feedback provided in audio file format?

• How do students use feedback when it is provided in audio file format to support their learning and how does this usage differ from their use of conventional written feedback?

• Is it practical and feasible for tutors to provide feedback to students using audio files?

Method

The study involved 15 student volunteers from Biological Sciences at Staffordshire University and two tutors (the authors of this paper). The students were studying human biology awards and comprised nine Level 2 undergraduates and six Level 3 undergraduates. Eleven of the students were full time and four were part time. Ethical approval to conduct the study was gained from the Faculty Research Ethics Committee.

Each student submitted a sample of their work for formative feedback to the tutors either as hard copy or as e-mail attachments. All the work related to modules that the tutors were currently teaching, but, within this pilot feasibility study, no restrictions were placed on the type of work that could be submitted. Work received included essays, parts of dissertations and written reflections.

After reading the students’ work, tutors recorded their spoken feedback on a desktop PC using Audacity (http://audacity.sourceforge.net/). The recorded files were then converted to mp3 format using Switch (http://nch.com.au/switch/index.html). Both of these packages are available as freeware. The mp3 files were then sent to the students as e-mail attachments.

The tutors, who were both experienced markers of student work, also maintained a reflective record of their experiences when providing the audio file feedback; in particular this record noted how these experiences differed from their usual experiences when providing written coursework feedback.

Semi-structured interviews of the 15 student participants took place within 3 weeks of receipt of the feedback. Typically, the interviews took place within one week, but this was not possible for some students because of their part time status and consequent employment commitments. The interview schedule (Appendix 1) comprised items concerned with, firstly, students’ perception of the feedback they had received (e.g. what they thought of the overall quality of the feedback and what they thought the tutor was attempting to say in the feedback) and, secondly, students’ utilisation of the feedback (e.g. did the feedback help give more meaning to the assignment and was the feedback helpful to their learning). The interviews took place in the presence of the two tutors and were deliberately informal and confidential. The tutors made contemporaneous notes and all 15 students consented to having the interviews audio recorded for transcription.

The qualitative analysis of the interview data involved clustering units of relevant meaning and identifying general and unique themes (Cohen and Manion, 1994).

Additionally, the written work provided by 12 of the students was conventionally marked. This was done at a later date and the written comments were added to the work without reference to the previously generated audio file. In these cases both the written feedback comments derived from conventional marking and the audio feedback comments were then classified using a scheme based on that of Brown et al. (2003). Quantitative comparisons of this data were made based on the total numbers of comments classified to each category for the audio and written feedback respectively. Paired two-tailed t-tests were used to compare feedback comments in individual categories with significant differences assumed when p<0.05.

Results

1. Interview Data

Overall, the students responded very positively to this type of feedback. All students said they would like to receive more feedback in this format, with two students showing a preference for both written and audio. They all judged the feedback to be of good quality with thirteen students stating their reasons for this as aspects of the nature of the audio feedback when compared to written feedback. Broadly this related to audio feedback providing more depth, perhaps related to it providing suggested strategies for solving problems rather than just stating what the problems were.

’comparing it to written comments where tutors say ‘could expand here’ and I think to myself ‘how… the (audio feedback) tells me areas perhaps I could expand… and I can think ‘yes I know what you mean… I understand how I can expand in those areas.

(Student 5)

Audio feedback provision was perceived by 12 students to be of greater quality compared to written feedback because it allowed students to be more aware of what tutors were trying to achieve.

’…with written feedback you have got to try and understand what the person is thinking as they are reading it (piece of work), but hearing them speaking you could see where their thinking processes were, you could hear the thinking processes’.

(Student 6)

Quality of feedback was also, in 14 students, related to implementing the feedback more effectively.

’…with written feedback it’s just a circle with a question mark and you’re thinking ‘what’s wrong with that’, but audio feedback they (tutors) tell you exactly what it was, if it was spelt wrong or if the wrong word had been used, or just it didn’t make sense’.

(Student 1)

All the students spoke in terms which showed some active participation with the feedback, 12 students spoke in terms of seeking understanding and recognising some parts of the feedback as being more important than others.

’You can actually interpret the importance(of audio feedback…with written feedback you can read it over again but it kind of gives you the same thing, but when you listen to it, you gain more of an understanding because of the tones of the voice and you get a feeling of how important (specific comments are) by the way they (tutors) talk… it was communicating…(the audio) meant more that just words on a piece of paper’.

(Student 2)

Thirteen students were unconcerned by the absence of written comments. Of the remaining two students, one said that they would have preferred both written and audio feedback while the remaining student expressed a preference for written feedback. All students listened to the feedback with a copy of their submitted work in front of them at least once and 12 of the 15 students altered their written work as they listened to the feedback on a second or third occasion. This was done either by making hand written notes on a printed copy of the work or by directly changing the text of the word-processed document using a PC. Of these all 12 said that they would work differently with the audio feedback, for example:

’I’d listen to it and write my own comments and then I’d go back through it a second time looking at the notes I had written for each paragraph… and I’d thought of things in my head that I could put (in)

(Student 10)

Pausing or re-winding the audio feedback, which was an advantage for 11 students, seemed to differ from stopping or re-reading written feedback

’You don’t need to pause written feedback (because) there wasn’t as much detail with written feedback… you just read it through’

(Student 11)

Eight students said that the tutor feedback gave some added meaning to the piece of marked work for example:

’yeah I would say that it provides you certain ideas that you wouldn’t have (thought of yourself). The way other people interpret things… you wouldn’t necessarily have (thought of yourself)’ (Student 9)

This student gave an insight into the other ways students seek perceptions of others and doing so may indicate the development of a community of practice (Wenger, 1998).

’We have started e-mailing work around because there are three of us; we are actually giving each other feedback which I found quite useful’

(Student 9)

This student was also able to put the tutor feedback into perspective

’I could get by without tutor feedback… it would be harder but we’d get through… it wouldn’t be as useful… the tutor sets the essay questions they know what they want from an essay so its better if you get (their) feedback’

(Student 9)

Nine students said they would use the feedback again when preparing work for different tutors, but all spoke of using feedback for similar assignments

’I think it will help with essays in different modules… pointers given about creating arguments in (this) essay and how to reference can help me next time I do an essay’

(student 13)

Thirteen students said that the feedback related to the module learning outcomes but only five students said that they used the learning outcomes when working with the feedback. Only one student mentioned marking criteria. Four students mentioned an inability to read tutors handwriting and one student spoke of ease in storing audio feedback compared to written feedback. One student commented that audio feedback appeared to indicate that tutor seemed more caring in giving audio feedback. Thirteen students listened to the feedback more than once with some doing so while they were doing other things such as walking to work.

2. Tutor Comments

The tutors were also positive about the use of this form of feedback. They were aware that they were able to provide more detailed feedback using examples of how the work might be changed within the same timescale as would be involved in providing written feedback. While in this study tutors found that providing audio feedback did not save them time, they felt it might do so with more practice. They felt that providing audio feedback to larger groups of students was feasible, its main benefit being that the feedback was of higher quality.

Pitfalls identified during the course of the work concerned the large size of the audio files generated (up to 11Mb) making them incompatible with some e-mail systems. Within this study audio files could not be e-mailed to two of the 15 students for that reason. In these cases the feedback was provided on disk.

3. Feedback Classification

The classification of the feedback produced is shown in Figure 1 where data is expressed as overall totals in each category rather than means because of the diverse nature and lengths of the types of coursework that were analysed in this study. The overall total number of comments was similar for the audio and written feedback (395 and 359 respectively) and only two comments (one audio and one written) were classified as “other” since they did not fit within the specific feedback categories. It was also apparent for both audio and written feedback that there was a deficiency of comments which were explicitly ‘justifying marks’ or ‘suggesting approaches to future assignments’; although this deficiency was less for audio feedback than for written feedback.

Figure 1 Classification of audio and written feedback comments for assignments that were independently marked by both means. Categories are based on Brown et al. (2003) and range from those which are very specific to the assignment on the left of the chart to those which are more generic on the right. The discrepancy between audio and written feedback for ‘identifying errors’ and for ‘demonstrating correct practice’ was statistically significant (p<0.05, paired t-test)

’Giving praise’ was the largest category for written feedback followed by ‘identifying errors’, ‘explaining misunderstandings’ and ‘engaging students in thinking’. For the audio feedback the largest category was ‘engaging students in thinking’ followed by ‘giving praise’, ‘demonstrating correct practice’ and ‘explaining misunderstandings’. The discrepancy between audio and written feedback for ‘identifying errors’ and for ‘demonstrating correct practice’ is particularly noticeable and also statistically significant. There was a preponderance of written feedback on the first category and a preponderance of audio feedback in the latter category.

Discussion

This preliminary study indicates that students perceive and implement audio file feedback in different and more meaningful ways than written feedback. Findings that a large majority (13/15) of students listened to the audio feedback more than once and that they (12/15) used it to annotate or alter their original work as they listened to the feedback do demonstrate that they did consider the feedback in some depth. Taras (2002, p506) commented

’Formative feedback is not complete until the students have produced an equivalent piece of work where the issues have been addressed and remedied, that is to say, until true learning has taken place and has been shown to have taken place.’

It seems that most students spontaneously began to complete the formative feedback process when the feedback was delivered as an audio file.

Not only did students take the feedback seriously, they seemed to appreciate the feedback more in the sense that all 15 students found it to be of good quality and several also indicated that they that found the feedback more personal or that it showed that the tutor cared about the work. Audio feedback, at one level therefore, appears to address some concerns raised be Carless (2006) regarding tutor ‘discourse’ through feedback.

Furthermore, both this study and our previous work (Orsmond et al., 2005) do indicate that students do value tutor feedback, but that some students could work independently of tutor feedback. This study also supports two findings of (Orsmond et al., 2006). Firstly, students actively seek feedback from a variety of sources. However, what this study is unable to show is whether there is any possible conflict between other sources of feedback and those from the tutor and furthermore, if conflict exists, what are the consequences of this for student learning. Secondly, this study supports the earlier finding that students believe that successful completion of a module is dependent on completing the form (such as essay or report) of assignments, rather then meeting the function of assignments (in this case the demonstration of learning outcomes). This form and function dichotomy was further highlighted by the limited discussion by students of learning outcomes, and their limited awareness of using marking criteria to better interpret the feedback and generate more meaningful outcomes of learning.

Thus, while there appears to be good student ownership of feedback, with evidence of original thought and awareness of understanding, the comments of student 9, still show students have a dependence on ‘what the tutors want’ rather than using criteria and learning outcomes to demonstrate their own learning. Perhaps tutors should look to students to justify the quality of feedback in terms other than the successful completion of individual pieces of coursework. The quality of feedback may need to be seen in terms of developing or benefiting individual student’s learning strategies. This could effectively be done by encouraging students to engage in self-assessment as defined by Boud (1995, p1) where self-assessment is seen in terms of both learning in the ‘now’ and also in the ‘future’. In doing this tutors may encourage students to work more effectively in terms of their learning perhaps by contextualising it within their professional development as biologists (Orsmond and Merry, 2007). Thus this study helps unpack the conclusion of Higgins et al. (2002) that the feedback academics provide may not be effectively used by all students. In this study, students are effectively using feedback, but only in a limited way.

In this context it is also worth highlighting the deficiency of ‘justifying marks’ or ‘suggesting approaches to future assignments’ comments; although this deficiency was less for audio feedback than for written feedback (Figure 1). It is possible that this paucity results from the fact that such comments may be considered generic and so only need to be said once, and this contrasts with categories such as ‘explaining misunderstandings’ where comments need to be made each time the event occurs. However, this lack of more generic comments may mean that the feedback scaffolding provided by tutors is biased towards learning in the ‘now’ rather than considering students’ development as professional biologists. To overcome this and to enable students to integrate the feedback they receive, the use of feedback diaries (Merry et al., 2000) may be beneficial.

The variation in audio file feedback may be another reason why students judged it to be of good quality. Pang (2003, p150) stated:

’we can only discern what varies… for instance, one would not be able to discern the aspect of gender should there be only one gender in the world; or the aspect of happiness should the Earth be filled with invariable happiness.’

One reason why students may actively seek feedback from a number of sources may be because this process generates variation through the different perception put forward. In the audio file feedback variation was observed by students in the tone of the tutors’ voice and they were able to use this to enable them to discern the most important aspects of the feedback. Such variation may not be conveyed in written feedback (Orsmond and Merry, 2007). In this study, students also commented that the audio file feedback was more detailed than written comments and this theme was also picked up by the tutors who found they naturally provided examples in their audio feedback of how the work might be changed. The use of examples may have provided additional opportunities for introducing variation. Lea and Street (1998) have commented that brief feedback comments from tutors often have little meaning to students.

These observations are consistent with the relatively high numbers of ‘demonstrating correct practice’ and low numbers of ‘identifying errors’ comments in the audio feedback (Figure 1) and may also have contributed to the higher numbers of ‘engaging students in thinking’ comments. Tutors felt they did not use many examples, and hence provided little variation, in written feedback was because of either time or space constraints for marginal comments. Additionally there was an appreciation that subtleties of thought, further opportunities for variation, that indicate merely possibilities for change (rather than direct instructions to change) are more able easily conveyed using the spoken word than in writing.

The high proportion of ‘giving praise’ comments which is particularly marked for written feedback (Fig. 1) is also worthy of comment. Hattie and Timperley (2007) in their meta-analysis have found that such comments had a low impact on student learning; a finding which the authors’ own unpublished observations support. This is to be expected because ‘giving praise’ comments do not lend themselves to be worked with by students to develop their learning by closure of the gap as described by Sadler (1989). The fewer ‘giving praise’ comments in the audio file feedback may have been a contributory factor to the more active way in which the students used this type of feedback.

While many findings of this study are very positive concerning the benefits of audio file feedback, it must be acknowledged that this is a pilot study based on a small number of self-selected volunteers who knew in advance that they would be asked to participate in an interview concerning their opinion of the feedback. As such the study group might be considered to be biased; however there was ample opportunity in the interviews for the students to state that the feedback did not meet their expectations if that was indeed the case. All 15 students said that they would like to receive more feedback in audio file format with only a small number (2/15) stating that they would like to continue to receive written feedback either alone or in combination with audio feedback. These factors lead the authors to believe that audio file feedback is an approach that can enhance student learning and which should be pursued in further studies.

It is also acknowledged that the “novelty factor” of audio file feedback may have encouraged students to pay particular attention to it in this study. This will be judged by comparison of these results with subsequent studies as students are given more feedback in this format.

Such future studies might investigate, firstly, the integration of audio file feedback into virtual learning environments such as Blackboard® and, secondly, the application of this type of feedback within summative assessments including its compatibility with institutional quality assurance procedures.

The use of a virtual learning environment for delivery of audio file feedback my serve to overcome the difficulties experienced in sending large files to students by e-mail. Additionally, it may also be possible to reduce the large size of the audio files by refining the recording process. However the authors have concerns that in doing so the sound quality might be reduced to such an extent that the personal aspect of the feedback as perceived by students might be lost.

Overall, it could be that audio file feedback is particularly influential to students’ learning because it meets many of the requirements for effective feedback outlined in Gibbs and Simpson (2004) including being detailed, promptly received (by e-mail) and understandable to students. Audio file feedback may be more understandable to students because they are more used to information being conveyed as sound than as written words possibly reflecting their increasing use of multimedia technology in their lifestyles and, perhaps mobile phones and mp3 players in particular. In terms of being understandable it is also worth noting that 4 of the 15 students interviewed reported that the often did not read written feedback because they found tutors’ handwriting difficult to read. Audio file feedback overcomes this illegibility.

Guidelines for tutors providing formative feedback to students via audio files

The findings of this study suggest that providing formative feedback to students via audio files is feasible and the following approaches are recommended:

• Use audio recording of sufficient quality to enable students to appreciate the personal qualities of your voice. A high quality microphone or headset with a USB connection is recommended and ensure that the recording level is not set too high as that will distort the sound.

• To keep file size as small as possible use mono recordings instead of stereo and set the sampling rate to a level suitable for voice recording which is lower than for music recording.

• Spoken feedback on individual pieces of work does not need to be supplemented by written feedback. It is sufficient to indicate verbally the points in the text to which the comments refer.

• Spoken feedback should not be presented in an excessively formal way. Students appreciate a more personal caring approach.

• Spoken feedback should not be rushed. Students may want to replay it anyway.

• Give examples of how the work might be changed in order to circumvent any general deficiencies that are noted. Suggesting a paper or a section of a textbook that can be read may also be helpful.

• Always try to be positive and give praise for good aspects of the work.

• Make explicit how the feedback contributes to the development of the student as a professional biologist.

Acknowledgements

The authors would like to thank the students for their willing participation in this study and the Centre for Bioscience for their financial support. Thanks are also extended to the referees of this paper for their helpful comments on the manuscript.