Submitting the ‘right’ reflection

ABSTRACT

While the benefits of reflection for students are well documented, there is a parallel body of research pointing to varying levels of engagement with reflection. Students on a four-year business information systems programme have to complete a reflection as part of their placement assessment. Using a constructivist qualitative methodology, this research explores the students' experience of reflecting with a view to identifying what supports could be useful to guide students in the future. Using semi-structured interviews, the students described a focus on description of activities and overall uncertainty on what to write. There are multiple possible interventions which could support the students but some would require significant and high-level support within the university.

KEYWORDS:

• Reflection
• placement
• assessment
• business information systems
• qualitative
• higher level education

Introduction

While higher-level education has a role to develop the student for the world, it also has a duty to prepare the student for the world of work (Davidson, 2017). Placement offers students ‘an unrivalled opportunity for situated learning’ within their discipline programme (Sheridan & Linehan, 2011). Reflection is a key part of the assessment strategy for established work placement programmes (Helyer, 2015). The process of reflection has been found to deliver multiple benefits for graduates (Branch & Paranjape, 2002; Helyer, 2015; Kolb, 1984; O’Donovan, 2006). Research suggests that students of reflection find it challenging and need supports (Moon, 2006; O’Donovan, 2006; Veine et al., 2020). This research explores the experience of Business Information Systems (BIS) students who complete reflection as part of their placement assessment and asks what can be done to support the students.

Literature review

There are multiple definitions available and an acknowledged need for a standard definition in order to provide clarity on how to reflect (Nguyen et al., 2014). A five-component model was developed based on a systematic review of the literature and provides a framework for understanding the wide range of interpretations on reflection.

• Thoughts and actions

• Attentive, critical exploratory and iterative actions

• An underlying conceptual framework

• A View on change

• The Self.

However, there is no mention of emotion or feelings in the final components of this framework. On examination, six of the fifteen authors considered in the review refer to experiences or feelings in their definition of reflection (Nguyen et al., 2014). In contrast, a much more emotionally aware model was developed based on a similar systematic review, which focussed on an awareness of feelings and discomfort and a critical analysis of feelings and knowledge (Atkins & Murphy, 1993). Sharing reflection can be expected to ‘daunting’ and can ‘cause feelings of vulnerability’ among students (Helyer, 2015). This issue of including feeling in our understanding of reflection is key to clarifying boundaries for placement students in understanding the expectations of reflection. Is reflection appropriate to share with industry and academic supervisors or is it something much more intimate? Drawing from the literature on assessment, Boud surmises that good reflection is something that a student would be embarrassed for a lecturer to read (Boud, 2001).

Benefits of reflection

Notwithstanding some issues with the definitions of reflections, there is widespread support from industry to build reflective skills in students in Ireland (IBEC, 2018). In a complex and changing world, reflection would support graduates in adapting and finding new roles (Helyer, 2015). Students also experience better integration of theory and real-world experience (Helyer, 2015; O’Donovan, 2006). In essence, the student becomes a better learner with the benefit of reflection on the placement experience (Kolb, 1984). At a higher level, reflection will enhance the ‘growth of the individual’ when it is implemented well (Branch & Paranjape, 2002).

Issues of reflection as assessment on placement

However, there are issues emerging. The reflective process should take place as a series of events rather than a singular event and should be embedded from early in the programme and repeated throughout (Veine et al., 2020). While there are many benefits to the process of reflection, there are concerns that in the context of assessment students sometimes sacrifice the benefits of critical reflection in order to gain marks (Birden & Usherwood, 2013; Lasen et al., 2018; Whitaker & Reimer, 2017). Similar behaviour was documented with Irish students adjusting their behaviour on placement assessment in order to score higher marks from their supervisor rather than a demonstration of their full skills set (O’Connor et al., 2018).

What we can do to support the students

There are some directions emerging on how students can be better supported to develop their reflective skills. These include a constructive alignment between the work that they are performing and the assessment mechanism, allowing the students to fully express the learning that they are achieving on placement (Ajjawi et al., 2020; Lasen et al., 2018). Feedback is required on the reflective drafts to further scaffold their reflective processes (Lasen et al., 2018). A trusted relationship with the academic supervisor is needed to allow the student to safely express themselves in their reflection in depth (O’Donovan, 2006). The use of an assessment rubric for reflection writing also helps students to focus their writing (Cheng & Chan, 2019).

Much of the reflection research has focussed on the nursing profession. However, student nurse placements are seen primarily as learning opportunities for the nurses and experienced practitioners on the placement are seen as mentors and teachers with a role to support reflection (Hayter & Jackson, 2020). In contrast, BIS placement students are paid employees typically and the experienced practitioner’s role is as their work supervisor rather than teacher.

The purpose of this research is to explore how the students can be supported to reflect by first understanding their experience of reflection as part of their placement assessment.

Methodology

Context

Students registered on a four-year level 8 programme in the BIS have an elective of Industry Placement. The elective is worth 30 ECTS credits for a 15-week placement. There are 80 BIS students who take placement each year. About 100% of students choose the elective and 95% secure an industry placement. About 98% of placement students are paid. The assessment mechanism for the placement module is made as follows.

• CV and Interview Questions Preparation 10%

• Presentation and Interview with Academic Supervisor 20%

• Reflection 20% (15 Weekly logs worth 5% and Final Reflection worth 15%)

• Industry Supervisor Performance Feedback 50%

The students anecdotally find it challenging to reflect while on placement. The training for reflection typically happens in December of the academic year before they go on placement, while they submit their reflection at the end of semester two typically in April. The reflections are of varying standards. The reflection is marked by an academic supervisor who may or may not be well known to the student.

Each week they submit a learning log on the Virtual Learning Environment (VLE). The student is asked to reflect on 2/3 learning events each week in the learning log and align their learning to the module learning outcomes. The industry supervisor reviews the learning log each week to ensure that it is an honest account of the work that was completed and that there are no company confidentiality issues in relation to the log. At the end of the 15 weeks, the student submits a final reflection to summarise their learning in relation to the learning outcomes for the module.

Design

The nature of this research is to uncover the students’ experience of completing a reflection for the purposes of assessment in order to better support the students in the future. It is clear that each student will have experienced reflecting in a different way and the research should reveal what the truth of that experience for each student. This accepts that each student will have their own truth and will have constructed their ‘own meaning’ and hence the research sits clearly within the constructivist paradigm (Gray, 2013). A phenomenological approach was taken to explore the ‘lived experience’ of completing the assessment (Larsson & Holmström, 2007). The focus was on the individual student and a qualitative approach was taken to gather the student experiences in the form of semi-structured one-to-one interviews (Gray, 2013).

Data collection instrument

Semi-structured interviewing was chosen as the data collection instrument to allow the participants to describe their unique view of reflecting for the purpose of assessment on work placement (Cohen et al., 2017). Open-ended questions allowed students their voice and more control in the interview (Norton, 2019). The semi-structured interview allowed the researcher to follow-up on answers with more probing questions (Hobson & Townsend, 2010; Newcomer et al., 2015).

Data collection

Each student was interviewed for approximately 20 minutes. An interview script was used at the start of the interview to settle the students and advise in relation to the informed consent form. A diagram of mainly open-ended questions was used to guide the interviews. Some follow-up questions were asked to explore specific answers and to clarify responses. A summary of the student’s comments was made at the end of the interview. Using a closing script, students were invited to add further comments through email and were thanked for their time.

Data management

The interview video calls were recorded and a transcript was generated using cloud-based software. These files were transferred to a secure cloud-based location provided by the university. All transcripts were reviewed and names of participants, their placement organisations or any identifying elements were removed. The transcript was reviewed against the video recording and appropriate manual adjustments were completed. All files were named using a coding system. The coding system file was password protected and stored in a separate location.

The notes made during the analysis stage including handwritten notes including handwritten notes at various stages were all stored in a secure cloud location in the interests of transparency.

Participants

The participants were six students from the Year 4 BIS class. This group has completed a reflection in the previous year for their work placement. They were chosen on a systematic basis to reflect a range in marks in the reflection component of their placement as well as gender breakdown. The participants include three male and three female students. There were three high scoring students and three low scoring students.

Analysis

The researcher had video footage as well as transcript files available. An immersion in the data as a first stage of the analysis was followed by development of categories and themes (Norton, 2019). The computer-generated transcript was reviewed first to anonymise. The transcript was reviewed again in line with the video footage to correct errors in the transcription. Following the immersion, the video footage was then reviewed making notes of items of interest (Braun & Clarke, 2006). The video footage was a rich source to support the phenomenological approach in contrast to reading the transcript. Re-watching the video footage brought the experiences more to life than reading the transcript. The transcripts were used to find the quotes and support the building of a table of themes. The themes table was then constructed around the themes with notes made for each students’ related comments. Themes were then re-examined along with a review of the video footage adding other students comments or noting that other students had not made any contribution to the theme. This was an iterative process considering what would remain as a theme, what was linked to another theme and were there other themes based on what had not been mentioned. This process was carried out in parallel with a personal reflective journal noting emotional and gut reactions to the data. The purpose was to build awareness of the subconscious reactions to the data and to enable a better understanding of the data (Erlingsson & Brysiewicz, 2017). Finally, the ‘story’ of how the students completed their reflections while on their work placement was written with compelling quotations, observed gaps, and a convincing argument for supporting interventions (Braun & Clarke, 2006).

Ethics

The interviewees were invited to take part in the research by the researcher and were provided with an informed consent form by email prior to consenting to complete the interview. A data management plan was described in the informed consent form for the students. Each interviewee consented to the interview. They were asked at the start of their interview if they understood the form and advised that they could terminate the interview if they wished. At the end of the interview, they were advised that they could come back to add more information if they wished. The researcher does not have any teaching or assessment responsibilities with these students in their ongoing studies Ethical approval was obtained from the University Ethics committee.

Bias

The researcher was previously an assessor on the work placement module for some of the students and acknowledges that this may have played a role in the students consenting to take part in the interview and in their subsequent answers. However, the advantage was the mainly positive response by the students and the familiarity that the researcher had with the technical language and placement structure (Berger, 2015). As the researcher is known to the interviewees, the researcher has accumulated their own ‘baggage’ in relation to the topic and the students (Hobson & Townsend, 2010). However, it is recognised that the key to minimising the bias is the acknowledgement and declaration of same (ibid). The researcher maintained a research journal and a reflective journal to minimise bias (Ortlipp, 2008). The researcher had previously undertaken unconscious bias training. The questions were reviewed by another researcher prior to use in a pilot interview. Subsequent to the pilot interview, the interview script, interview transcript, and questions were reviewed to ensure that no leading or biased questions or remarks were used.

Results

The key theme that emerged was that students put their energy into gathering data to support a report about their activities. This theme is evidenced by their own description of weekly activities and their conversations with peers. A supporting theme is that they are uncertain on content and length of reflections. Some students can see value emerging from the reflective process even in its limited form.

Weekly activity descriptions

The students described a weekly activity of writing about their activities, which involved some preparation work on their behalf.

S1’ So I suppose even during like the week I was very conscious of preparing myself mentally that I’ll have to do this at the end of the week. So taking those screenshots, even if I saw something new on an email, how people communicated or something, I really focused on the weekly log….so you could back up what you were saying’

S1 ‘on Friday … I said I’d look back on the week I’m really look and focus on what I learned and what I developed and how that all that tied into my life skills being hard or soft and what I actually developed outside those skills that I already had..’

S2 ‘So I suppose my process for that would be if I couldn’t remember, My emails basically had all of my workload and I would be able to refresh my memory on what I had done in previous weeks’

S3 ’ I would take a note just so that I remember this, but in general if everything is kind of in your calendar to remind you anyway.’

S4’ Taking those little notes that I had made from earlier in the week and putting them together and seeing the best ones that I saw fit the couple of moments and once that was all put together, clear it up with my Manager’

S5 ‘I just take that down, jot down like in a note and then at the end of the week just kind of put it into proper words they’re looking for ‘ ‘wasn’t going off the top of my head’

They all completed their weekly logs while at work on a Friday and this was driven by the need to get their work placement supervisor to sign off on the logs at the end of each week.

Uncertainty

There was uncertainty about the reflection in terms of length.

S2 ‘How long are yours and all the other things that come with it, so I’d say it’s literally like any other assignment we’ve had in college’.

S6 ‘I was asking people like how long is yours? I feel like mine is too short. What will I do?’

However, their focus on activity meant that when they had repetitive weeks of work activities, they were unsure about what to put in for a week’s log. They consulted with class colleagues to get clarity. These consultations were illuminating in terms of highlighting what was on their minds during the writeup of the weekly learning logs. To give context to the quotes below, some of the students described questions that they asked of their peers, while others described questions that were being asked by peers.

S1 ‘always reaching out to them on a Friday being like what aspect you know, what kind of perspective you taking from this. Need to change it up from last week’.’What can we say, give me some ideas’

S2’ I found them easy enough to write as well because I did have a lot of new things that I was doing’

S4 ‘It was kind of if you were struggling you might ask them all Yeah, what have you put down because I know they might have interpreted something and you wouldn’t have even thought of making a point of it and ask what have you put down this week’

S5 ‘Got busy and stuff repeated didn’t know what to write after first few weeks’

The ‘right answer’

They didn’t see the reflections as being about themselves and how they had changed. Instead, they were still looking for a ‘right’ answer. So they described an uncertainty about what to say.

S3 ‘no active feedback so you have to submit and hope for the best’

S6 ‘Afraid I wasn’t putting in the right thing or telling you what you wanted to hear’

S2 ‘what are you writing here? Because I don’t know what I’m meant to be writing. … I suppose you just be looking for a bit of clarification to make sure that theirs is similar or how it should be compared to everyone else is’.

S1 ‘its not like an exam where we knew exactly what ye were looking for’

Learning

However, there was reference by some students to a change and to the sense of self.

S1 ‘It makes you step back I think, and really take a broad view of ok …What did I actually learn? Because if you don’t think about it, you’re never gonna think about it I don’t think, and especially that typing it up or putting down to paper really makes you emphasize OK. Actually I did learn something I did develop.’

S2 ‘It did force you to I suppose evaluate what you have learned more than I suppose if you didn’t have to do this process, and because you actually need to be writing it down and actually saying, OK, what did I do? Here what do I learn here? It was different from last week. So it obviously made me realize what I had learned more probably than if I didn’t have to do it.’

Some students understood the value of the logs citing their use for graduate interviews. They also used the logs to support them on writing their final reflection.

S2’ Just really what I wanted to do was get down my main learnings into that final reflection and show what I had done and what I had learned’.

However, it was a slow realisation for some when it was only at the end of the 15 weeks that they realised the value of the logs. For two students, it was a slower dawn.

When asked about retaining weekly logs on the placement assessment the students replied ‘if you asked me 15 minutes ago I would have said no to weekly logs but realised throughout interview – key to doing well in reflection’ S2.

Five students were keen to keep reflection and the associated logs as part of the placement assessment mechanism. One of the five suggested splitting the weighting to give 10% to the logs and 10% to the final reflection. Only one student suggested removing the weekly logs as they were ‘too monotonous’.

What they didn’t say

While the students were not asked explicitly about a model that they used, there was no reference to use of a model in their description of how they did their reflection. Gibbs’ model and Moon’s level of reflection had been covered in preparatory work. Equally, students did not describe any level of discomfort at disclosure of personal information or suggest any tension with trust on personal details of their reflection.

Discussion

The students are not fully engaged in the process of reflecting as described by Nguyen et al. Their reflections are typically describing activities rather than about their thoughts. There is little evidence of awareness of themselves changing. For some students they saw the sense of self changing but the key theme is coming back to an account of what they did. There was no mention of a framework being used to support their reflection process. So in relation to the Nguyen model described earlier, they are overly focussed on actions (Nguyen et al., 2014). They are focussed on the description of the experience rather than the learning from reflecting on the experience (Dewey, 1960). This is in line with other studies who found that early practitioners of reflection are focussed more on description rather than on critical reflection (Moon, 2004; Veine et al., 2020). It may be that their understanding of learning has been based within the classroom environment, which has been more teacher centred than student centred and more behaviourist than constructivist. There is a sense that the students are still seeking the ‘right’ answer in a belief that there is a right answer for a reflection. Their placement is most likely the first significant authentic learning in their degree programme and this is their first reflection. In changes to student centred learning, students can experience feelings of abandonment, isolation, and indeed being frightened (O’neill & Mcmahon, 2005). This sense of fear and abandonment can be seen in the comments above referencing ‘no active feedback’ and ‘afraid I wasn’t putting in the right thing’.

In other research, there was evidence that students were making decisions about creating content to avail of the best mark (Birden & Usherwood, 2013). However, all the students in the study were unable to remember a mark. Instead, they had only a memory of the employer or overall mark. There was no discussion of embellishing their answer for higher marks from the students much more that they were seeking the right answer and any content to add.

How can the student be supported?

For those students who scored highly in the reflection assessment, there was a greater account of the benefits of reflection in terms of self-awareness and use for graduate interviews. For the other students, they are missing the full set of benefits of reflection. The question is what we can do as lecturers to support student to engage in the full experience of reflection.

Start earlier

The students have not completed any reflection before now so there is a need to start the reflective writing much earlier in the programme is small steps to build their confidence and skills. This supports other findings which recommend to inclusion of reflection across a programme (Veine et al., 2020).

Staff training

The inspection by the industry supervisor is now a key driving point for students who prepare the learning logs for the supervisor to review as part of the weekly activity and this is supporting the ‘evidence gathering’. While the module and language has changed since the inspectorial history, the legacy of a burden of proof remains. Successful reflections have a link to the teaching staff and their own experience and practice of reflection (Veine et al., 2020). The focus of the reflection needs to shift from evidence gathering to a deeper understanding of reflection. This will also require a shift for staff as the language of logs and checking has remained.

Prompts

This difficulty with what to write points to a need for further guidance for students to allow them also to move closely to more reflective writing addressing the issues of self and change and other lenses. None of the students interviewed referred to any theoretical framework. The models had been covered in preparation work but were perhaps too early in the students’ reflection journey. A timelier online delivery in addition to prompts would be useful for students to link the frameworks with their reflection practice. In the nursing space where reflection is more mature, dedicated journals are now available which guide the nurse practitioner in the reflective writing offering stimulating questions, a reflection exemplars, reminder of Gibbs model all within one book (Coombs, 2015). The Employability Journal is a contender in this space with its focus on the awareness of employability skills developed on placement and other learning experiences (Bassott, 2017). There are other prompts available to get students started with reflective writing such as six minute writing, use of poetry, use of metaphor, writing from other perspectives, and sharing with peers that could be useful for these students in terms of shifting from the evidence base (Bolton & Delderfield, 2018). However, there is an opportunity for further research involving students to examine what other perhaps more contemporary prompts might work for that cohort.

Allow for free writing as well as submission

There was no evidence of any tension of disclose of sensitive or personal information when writing reflections for assessment by the students interviewed suggesting that there was no consideration of any sensitive material. There is a debate about the intimacy of what to write in a reflection. A solution lies in inviting the students to complete a private journal which will support them in submitting their learning (Boud, 2001). It is the process of reflection that is the outcome that is desired in students so an ability to reflect in the privacy on their own story should be encouraged with them having the choice about what to share in the assessment submission.

Peer support

There was consultation in the main among the group on what was being written about for each week with only one student mentioning contacting a lecturer when they were in difficulty. Students value peer support particularly when they are remote and it can help with reflection (Duke et al., 2015). Another study examined the use of peer support group to enhance reflection on practice and found that the benefits outweighed the negatives (Bold, 2008).

Be directive in how to reflect

The students approached the reflective activity as they would any assignment setting aside times at their computer. However, further advice in terms of setting a place aside that is different to allow for a different way of thinking and to allow for the switch off from work could be useful for students as well as a shift to written rather than typed work (Bassot, 2015).

Engagement with rubrics and feedback

An assessment rubric had been provided and this was consulted by the higher-scoring students but the lower-scoring students were unaware of the existence of the rubric. An explanation was given about what a rubric was and what it looked like as students looked blankly when asked if they had consulted the rubric. It was a year since they had completed the assignment but that said they were very clear on other details of how they had completed the reflection. This indicates that a deeper engagement with the rubric is required and timing is a consideration here also. There was some engagement activity with the assessment rubric but it was months before the students completed the assessment. The deeper engagement could include co-creation of the rubric, incorporation of exemplars, self-assessment, and feedback discussion (Dawson, 2017). Feedback on their progress for the purposes of improvement is very much desired by the students in the study. The combination of the feedback with the assessment rubric would support the students in the development of their own judgement on the quality of their reflection (Tai et al., 2018).

Implications for support interventions

There are multiple-support options available to lecturers ranging in complexity from an implementation perspective. A change to multiple reflections requires a change across the whole programme assessments mechanisms and agreement of multiple lecturers. Staff Training requires a financial and time commitment on behalf of the department and the lecturing team. A reflection journal requires a change in thinking and a financial commitment. Peer support requires additional staff time. Being more directive is a low commitment change that can be made. Change to rubric use and feedback require additional staff training and staff time to implement. These changes are not impossible but it is important to ground the discussion of their use in the practicalities of higher education. To overcome these difficulties there needs to be further and higher conversation on the importance of reflection with all its difficulties for all our graduates. This conversation will support the changes that need to be made so that reflection is fully supported rather than added to the curriculum as lip service.

Limitations

While it is acknowledged that this is a small scale study with six students in a one discipline, there is evidence to support other findings that more needs to be done to shift students from the descriptive to the deeper levels of reflection (O’Donovan, 2006; Veine et al., 2020)

Conclusion

While some of the students in this study recognise the value of reflecting on learning for their future graduate interviews, overall the students felt unsupported and unclear on how to do the reflection. They focus on recording their actions. Much can be done to support them to reflect in providing earlier reflection opportunities, staff training, supported prompts, personal journals, peer supports, guidance, rubrics, and feedback. However, these have financial and lecturer impacts and need to receive a full support from the university. Otherwise, reflection will continue to appear on curriculum but won’t have the opportunity to deliver the benefits for the students.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

C. Murphy

Catherine Murphy is a Senior Lecturer and Placement Manager in Munster Technological University, Ireland. Her research interests are in assessment and feedback and student placement experiences.

T. O’Mahony

Dr. Tom O'Mahony is a Senior Lecturer in Munster Technological University, Ireland. He lectures in the Department of Electrical and Electronic Engineering. and manages and lectures on the Master of Arts in Teaching and Learning in Higher Education. His primary research interests relate to higher education and include teaching and learning, assessment and feedback and engineering education.