https://orcid.org/0000-0002-2662-1919
ABSTRACT
Introduction: Doctors and medical students in the UK are currently required to provide evidence of learning by reflective writing on (among other things) feedback from colleagues. Although the theoretical value of reflecting-on-action is clear, research is still needed to know how to realise the potential of written reflection in medical education. This study arose out of efforts to improve medical student engagement with a reflective writing exercise. We used realist methodology to explain the disinclination of the majority to do written reflection on workplace feedback, and the benefits to the minority.
Method: Realist evaluation is a suitable approach to researching complex interventions which have worked for some and not for others. Focus groups were held over a three-year period with year 3 and 4 students. Focus group transcripts were coded for context-mechanism-outcome configurations (the realist approach to analysing data) explaining students’ choice not to write a reflection, to write a ‘tick-box’ reflection or to write for learning. A sub-set of eight students’ reflections were also analysed to ascertain evidence of learning through reflection.
Results and discussion: 27 students participated in 4 focus groups. Three summary theories emerged showing the importance of context. Firstly, written reflection is effortful and benefits those who invest in it for intrinsic reasons in situations when they need to think more deeply about a learning event. Secondly, following a reflective feedback discussion writing a reflection may add little because the learning has already taken place. Thirdly, external motivation tends to result in writing a ‘tick-box’ reflection.
Introduction
Medical graduates in the UK are currently required to provide evidence of learning by maintaining a portfolio including their reflective work [Citation1]. This is because reflection is deemed to improve self-awareness, ability to cope and to identify actions needed [Citation2]. The skills of critical reflection need to be developed and this doesn’t 'just happen',, so medical educators have been trying to help [Citation3–6]. Guidance for UK medical students on how to reflect as part of their practice has been developed by the General Medical Council with the Medical Schools Council [Citation7]. Although other forms of reflection are also recognised, the encouragement to write is clear in statements such as: ‘A reflective note does not need to describe full details of an experience. It should capture what you have learned and any planned actions arising from the experience’ [Citation7,p.3].
Written reflection can be useful especially if guided [Citation8]. Medical students volunteering to write a daily reflection on their paediatric neurology learning goals for a two-week block reported a positive learning influence in 85% of respondents [Citation9]. The quality of reflective writing (and possibly reflective ability) can be improved by various educational interventions [Citation10]. Bolton and Delderfield in their guide to writing and professional development use the term ‘write to learn’, describing this as a structured and supported process which ‘can be reflective and reflexive if students are facilitated towards critical attitudes and expected to write reflexively’ [Citation8,p.63]. Rather than being merely a record of what has been thought, reflective writing ‘is the reflective process’ during which ‘sense is made of the muddle of stuff in our minds’ [Citation8,p.136].
Written reflection is however not always popular among medical students when it doesn’t align with their learning preferences [Citation11] and they may dislike their reflective activity being assessed [Citation12]. If it feels like surveillance, practitioners and students don’t choose to share the experiences which most need reflection [Citation13]. GPs and trainees surveyed also had mixed opinions of the value of written reflection with a dominant perception of wasting time [Citation14–16].
Although the theoretical value of reflecting-on-action is clear [Citation17–19], the dislike of exercises in written reflection makes us question whether they are indeed adding to our other educational interventions to produce more reflective practitioners.
Maximising the potential of the portfolio to promote reflective practice is a holy grail of continuing medical education in the UK and elsewhere, with the call to research its variable success and to improve its usefulness still seeming to need answers [Citation20]. From our own experiences of implementing a system designed to promote learning from workplace feedback, we also found ourselves asking when and how written reflection can add value.
Environment for this study
With a firm belief in the importance of reflecting on feedback in order to learn, we have devised an online ‘learning from feedback’ system for students in a UK medical school. GP supervisors observe their students consulting with patients and give them one-to-one feedback formally on three occasions during each of the Year 3, 4 and 5 GP-based blocks (blocks lasting four weeks in Years 3&4 and 10 weeks in Year 5). The GP supervisors use the ‘Workplace Assessor’ web app [Citation21,Citation22] to capture a written summary of feedback discussions with their students. These formative assessments are mandatory. Students are then invited by automated email to use the online ‘learning from feedback’ form to capture their main learning points and action plans (see Appendix A for the invitation and online form). The ‘learning from feedback’ add-on is optional. If the student uses it, the GP supervisor gets a copy of their students’ reflections.
In 2015 as we launched this system, we obtained ethics committee approval for an action research study to evaluate and improve it. Student usage of the optional ‘learning from feedback’ reflective add-on to workplace feedback was monitored for all Year 3 and 4 medical students. Because usage was low in the first year (11.3% of feedback summaries were reflected upon in the first 5 months from launching the system), student focus groups were held to improve the system. Researcher AM designed and modified the system and did the monitoring. The changes made at that stage did not alter subsequent student use of the reflective add-on (see for monitoring data). An extension of the study for a subsequent cycle of evaluation to understand this was granted by the School of Medicine Ethics Committee ref ERP1260. Our research aim was to explain what it is about the ‘Learning from feedback’ system which is working or not working for students and why. Through gaining new understanding, such educational systems can be better designed.
Figure 1. Year 3 and 4 student usage of the ‘learning from feedback’ reflective add-on to their workplace assessment
Methodology and methods
The evaluation design we chose was to ask student focus groups to consider what might explain the low uptake of the system by their cohort of students, and to explain the outcome of being invited to use it, in their own experience. Focus groups were chosen because they can enable participants to voice their thoughts, and by drawing out differences, explanations can be crystallised [Citation23].
A realist evaluation approach to data analysis was taken in this action research, to build theory about what matters in designing similar systems. Realist evaluation is a suitable approach to researching complex interventions which have worked for some and not for others. In realist evaluation the researcher tests and develops theory, starting with an initial theory about how a programme works. The initial ‘programme theory’ for this system of learning from workplace feedback by reflective writing was derived from a literature review about effective feedback for the clinical trainee [Citation24] and posited that thoughtful writing of learning points and an action plan would create a virtuous loop in the feedback relationship, enhance and consolidate learning and make it lasting [Citation2,Citation8].
Realist evaluation produces transferable explanations by making explicit how context influences the outcome of an intervention using the concept of mechanism. Mechanisms are underlying changes in the reasoning and behaviour of participants that are context-sensitive [Citation25]. An understanding of mechanisms can come from asking participants to explain what is causing their behaviours (i.e. which mechanisms (M) are being triggered by which aspects of the system (C) to what effect (O) [Citation25–28]. The discovery of the same links occurring between context mechanism and outcome in the explanations of different participants (termed recurring CMO configurations or CMOcs) enables the programme theory to be developed and refined. Development of middle range sub-theories which incorporate context enables transfer to similar contexts elsewhere.
All Year 3 and 4 students on their GP block in June 2016 were invited to participate in a focus group. Year 3 students on GP placement in May 2018 and Year 4 students on their GP blocks in November and December 2018 were also invited. The focus group topic guide was developed to answer our realist research questions. It was the same for all focus groups and can be seen in Appendix B. Focus group facilitators were faculty members from the medical school who were not directly involved in this workplace assessment and feedback system. The focus groups were audio recorded and rendered pseudo-anonymous by substitution of identifiers at transcription.
Each focus group transcript was analysed by two of three researchers (JL, BW and SS) using realist evaluation methodology [Citation25,Citation28]. For this analysis, the reported elements leading to students using the ‘learning from feedback’ app or not using it were sought, with explanations. We were particularly interested in data containing explanations about decisions to use it or not, but also about whether and how learning from feedback actually occurred. The links in these explanations between context and outcome with the reasons given were coded as CMOcs. Coding was done independently and then compiled with discussion, comparing coding of the same text and collating similar CMOcs into middle range theories to build up a picture of the main ways this system was working (or not) for whom and why. A middle range theory in realist evaluation is a judgment about the repeatability of one or a set of related CMOcs. The analysis process is to bracket CMOs together then search for what it is about the contexts which is common. A middle range theory is a theory that is at the correct level of abstraction to be ‘useful’ and ‘testable’ [Citation29].
In initial analysis, the theory that writing should enhance learning appeared to be contradicted. Additional data was considered helpful to test this theory further. Mixed methods are usual in realist evaluation so we asked for additional ethics approval to look at the reflective writing of our next set of participants. All consenting 2017–18 focus group participants’ reflections were therefore analysed to ascertain evidence of learning through reflection. Three of the researchers (JL, BW and SS) read each reflection and classified them independently by depth of reflection. The classification was: descriptive text/some reflection/in-depth reflection and notes were mad to justify each classification. These notes were then compared, and consensus reached by discussion.
In the final programme theory, middle range theories about how context affects outcomes were used to explain the monitoring data and the evidence of learning through reflection.
Results
This section reports
analysis of focus group data explaining the use made of the system by students, what was working for them (or not) and why.
analysis of students’ reflective writing testing the theory that writing should enhance learning.
Analysis of focus group discussions
Nine students from Year 3 and 4 participated in focus group 1 in June 2016. Five Year 3 students participated in focus group 2 in May 2018, Seven Year 4 students in November 2018 and six Year 4 students in December 2018 participated in focus groups 3 and 4 respectively. 13 of the 27 students were female.
All but one group contained a mix of participants who had and had not used the reflective add-on when invited to. In focus Group 1: four of nine had used it; focus Group 2: two of five had used it; focus Group 3: all seven had used it; focus Group 4: two of six had used it. Despite the differences between groups, discussions about reflective writing and explanations about how students learned from workplace feedback were similar in all four focus groups and the main findings of analysis were consistent.
Three outcomes were described by students: not using the reflective add-on; using it to ‘tick a box’; and using it for learning. The 13 recurrent CMOcs (middle range theories) are presented under these three headings ().
Table 1. CMOcs with illustrative quotes
Explanations for students not using the reflective add-on
Some students felt that although they were reflective individuals, they preferred to reflect in other ways than writing so would not write unless they had to. This preference was described by some as innate and by others as learned through poor past experiences of mandatory reflective writing exercises (CMOcs 1–2).
Although these students had all consented to participate in this study and might therefore have been expected to have an interest in the system under study, some had not read the emails inviting them to reflect on their workplace feedback and others felt that competing pressures on their priorities moved this activity to the bottom of the pile. They described a barrage of medical school emails which they tended to neglect unless marked urgent (CMOc3).
Students had brief immediate verbal feedback from their GP supervisor after each consultation, and also longer discussions before their workplace assessment was completed by the GP. Having already discussed and resolved their internal queries by the time the invitation to write a reflection came there was no felt need to do further reflection (CMOc2). Also some students felt that only the most challenging events deserved a written reflection (CMOc4). For some, delay in receiving the workplace feedback summary made it difficult to recollect the learning events (CMOc5).
Others reported the feedback summary was not useful to them for various reasons, such as not having been observed by their assessor or the feedback not meeting their needs. They then felt they had nothing on which to reflect (CMOc6).
Explanations for students writing ‘tick-box’ reflections
Some students who did use the ‘learning from feedback’ system claimed little perceived value other than to satisfy others. They also described previous mandatory reflective writing exercises as ‘ticking boxes’ (CMOc7) and felt this could result in dishonest or edited reflection (CMOc8).
The subject of reflection being feedback from the GP supervisor rather than the student’s own choice of topic also promoted extrinsic motivation rather than intrinsic. Reflection was consequently felt to be less useful (CMOc9).
Explanations for students getting added value from written reflection
Students who got added value from reflective writing in the past or on this occasion did speak up as a minority opinion in each focus group. Some found writing a reflection laborious compared to reflective thought or discussion but had found that it could be useful to them. Students who found it beneficial were sometimes surprised that it was and explained that when they got around to it the cognitive process involved in writing did organise and reinforce or add to their learning (CMOcs10&11).
A felt need to reflect was described as being triggered by situations which induced internal conflict (CMOc12). This was not usually felt to be the case after workplace assessment feedback partly because if it did, resolution had already occurred during feedback discussion (see CMOc4).
The enthusiastic supervisor could however trigger useful reflective writing if their feedback conversations encouraged reflective thought and writing and also by reading and commenting on the student’s reflections (CMOc13).
Analysis of the ‘learning from feedback’ student writings
Fifteen student reflections were analysed (those of eight students consenting to analysis among eleven participants in 2018 who wrote reflections). Five were felt to show learning through in-depth reflection with evidence that the students valued the feedback, planned appropriate actions and focused on identifying why they received the specific feedback. Eight contained evidence of action already taken, action planning and some reflection and two were purely paraphrasing and summarising the feedback. This aligned with CMOc11.
Discussion
Summary of principal findings
For each outcome (not using the reflective add-on; using it to ‘tick a box’; and using it for learning), we have explanations which can help the understanding of outcomes in other studies of written reflection. We have consolidated the main recurring CMO configurations () into three summary theories or sub-theories of our final programme theory.
Theory 1: written reflection is effortful and benefits those who invest in it for intrinsic reasons in situations when they need to think more deeply about a learning event
CMOcs 10–13 support this and CMOcs 1–6 explain why some students in our study did not feel that their GP feedback was suitable material for such added effort.
Theory 2: there are contexts (such as following a good feedback discussion) when writing down learning points and action plans may add little value because there is no longer a need to reflect – the reflection has already taken place
CMOcs 4&5 suggest that medical students usually value their feedback relationship and feedback discussions and also add an explanation to a previous study [Citation29] which found that verbal feedback was often preferred to written feedback. The explanation may include that during the feedback discussion reflection is occurring and questions are being resolved.
Theory 3: external motivation is the main mechanism triggering the writing of a ‘tick-box’ reflection with little other perceived value. The word ‘perceived’ is used intentionally and we would qualify this theory by the finding of our analysis of reflective writing and the explanations of a minority of students that a ‘tick-box’ reflection may have some value
After the first focus group we considered making the reflective add-on mandatory (the suggestion of one focus group participant), but the reactions of the rest of that focus group indicated this might be counter-productive, as was confirmed by subsequent focus groups. CMOcs 7–9 and 11 explain how external motivation to write can diminish perceived value and may cause editing or embellishment of true thoughts.
The intended benefit of writing to organise, consolidate and extend learning was a minority outcome for students in this study. Students who experienced external motivation to write about their learning were unlikely to claim these benefits. However, our study of student reflective entries did demonstrate evidence of paraphrasing or summarising of feedback, logging action already taken, action planning and some reflection. This could be evidence that writing does improve learning and this may be occurring without students realising.
Interpretation of the results
We have applied these theories to our monitoring data. A schematic interpretation of our final programme theory explaining the use students make of written reflection on workplace feedback is illustrated in .
Figure 2. Final programme theory
Strengths and weaknesses
This research project was designed to improve learning from feedback. Although it did not increase use of the system it did uncover important explanations. Using realist evaluation makes these explanations potentially transferable to similar contexts. Two data types were integrated to support inferences. The study is of a real system but in a single institution. It could have been improved by including more participants who have found reflective writing to be helpful. This might have been achieved by collaborating with other institutions with more success in the use of portfolios. Discussing their reflective writings with students who perceived no benefit might have been a source of fruitful additional data.
Comparison with existing literature
Reflection requires a stimulus to be genuine, and it has been found that a meaningful encounter or a surprising ‘teachable moment’ would trigger intrinsic motivation – the felt need to reflect [Citation30]. Thompson et al found that if there was no cognitive or emotional dissonance, students didn’t reflect on the activity [Citation31]. Our findings may cast light on why reflective writing about workplace feedback is not always felt to be appropriate. Students who have had a good discussion of their consultation skills and have noted what they need to improve do not feel the need to consolidate this learning by writing. Nor do students whose feedback has not made them think.
External motivation caused some students to use the ‘learning from feedback’ system but for the wrong reasons and this may explain why they did not perceive benefit even when learning may have been taking place. Even though students knew that their reflective writing was not being formally assessed, their reactions were similar to those in studies where reflective writing is assessed [Citation12,Citation32]. Our findings may cast a different light on student beliefs that reflective writing does not add to learning [Citation11]. The suggestion is that learning is occurring but not perceived. This might fit Ross’ description of the transformative potential of wearing the ‘mask’ of the professional persona when participating in such ‘rituals of confession and compliance’ [Citation13].
Conclusions: implications for research and practice
Our conclusions are reflexive as we feel that it is still possible to improve our system now that we have understood the way it is working. The realist perspective also enables us to offer suggestions for the wider community of educators wishing to develop healthcare professional trainees into reflective practitioners. The context in which learners are being encouraged to reflect matters, and might be adjusted in the following ways:
In designing systems to facilitate learning from feedback can we trigger the important mechanism of intrinsic motivation? Exercises in written reflection could be better framed (to make students perceive benefit to learning as ‘Writing precedes thinking, not the other way round’ (8 pg. 138) and better targeted to need by being more student-led, both in timing and topic.
Writing may not be the best way to reflect on workplace feedback. Making the mode of generating reflection more flexible might help, including organising reflective discussions with peers.
Can we enhance clinical supervisor enthusiasm for supporting students to do useful reflections and frame feedback to students as questions to trigger deeper thought?
There are still possible benefits to students of writing down learning points and action plans, such as clarifying and consolidating learning, and providing a record. The framing of this exercise will be important, however, as the need to reflect is missing as the prime driver.
Acknowledgments
We are grateful to the participating students for helping us to understand what was and was not working for them in our feedback system. We also acknowledge the School of Medicine’s encouragement to develop this system of workplace assessment and feedback, and the support of the school’s Medical Education Research Group for our research.
Disclosure statement
No potential conflict of interest was reported by the author(s).
References
- General Medical Council. Outcomes for graduates. GMC publications; 2018.
- Academy of Medical Royal Colleges, COPMeD, GMC, Medical Schools Council. The reflective practitioner guidance for doctors and medical students. 2019.
- Schon D. Educating the reflective practitioner. towards a new design for teaching and learning in the professions. Oxford and San Francisco: Jossey-Bass; 1991.
- Driessen E, Van Tartwijk J, Dornan T. Teaching rounds: the self critical doctor: helping students become more reflective. BMJ. 2008;336(7648):827–830.
- Aronson L. Twelve tips for teaching reflection at all levels of medical education. Med Teach. 2011;33(3):200–205.
- Mann KV. Reflection’s role in learning: increasing engagement and deepening participation. Perspect Med Educ. 2016;5(5):259–261.
- General Medical Council. The reflective practitioner - a guide for medical students [Internet]. 2019 [cited 2020 Apr 26]. Available from: https://www.gmc-uk.org/education/standards-guidance-and-curricula/guidance/reflective-practice/the-reflective-practitioner—a-guide-for-medical-students
- Bolton G, Delderfield R. Reflective practice. 5th ed. London: SAGE Publications Ltd; 2018.
- Larsen DP, London DA, Emke AR. Using reflection to influence practice: student perceptions of daily reflection in clinical education. Perspect Med Educ. 2016;5(5):285–291.
- Vivekananda-Schmidt P, Marshall M, Stark P, et al. Lessons from medical students’ perceptions of learning reflective skills: a multi-institutional study. Med Teach. 2011;33(10):846–850.
- Grant A, Kinnersley P, Metcalf E, et al. Students’ views of reflective learning techniques: an efficacy study at a UK medical school. Med Educ. 2006;40(4):379–388.
- Sandars J. The use of reflection in medical education: AMEE Guide No. 44. Med Teach. 2009;31(8):685–695.
- Ross J. Traces of self: online reflective practices and performances in higher education. Teach High Educ. 2011;16(1):113–126.
- Curtis P, Gorolay S, Curtis A, et al. What do general practitioners think of written reflection? A focus group study. Educ Prim Care. 2016;27(4):292–298.
- Curtis P, Taylor G, Riley R, et al. Written refection in assessment and appraisal: GP and GP trainee views. Educ Prim Care. 2017;28(3):141–149.
- Curtis P, Booth P, Frankland S, et al. Written reflection in NHS appraisal: time for an alternative. BJGP. 2018;68(July):316–317.
- Schon D. The reflective practitioner: how professionals think in action. New York (NY): Basic Books; 1983.
- Kolb DA. Experiential learning: experience as the source of learning and development. Englewood Cliffs (NJ): Prentice-Hall, Inc; 1984.
- Mezirow J, Brookfield A, Candy L. Fostering critical reflection in adulthood: a guide to transformative and emancipatory learning. Jossey-Bass Higher Education Series, editor. Michigan: Wiley; 1990.
- Driessen E, Van Tartwijk J, Van Der Vleuten C, et al. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007;41(12):1224–1233.
- Lefroy J, Thomas A, Harrison C, et al. Development and face validation of strategies for improving consultation skills. Adv Heal Sci Educ. 2014;19(5):661–685.
- Lefroy J, Roberts N, Molyneux A, et al. Utility of an app-based system to improve feedback following workplace-based assessment. Int J Med Educ. 2017;8:207–216.
- Stalmeijer RE, Mcnaughton N, Van Mook WNKA. Using focus groups in medical education research: AMEE Guide No. 91. Med Teach. 2014 July 29;91:1–17.
- Lefroy J, Watling C, Teunissen P, et al. Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education. Perspect Med Educ. 2015;4(6):284–299.
- Wong G, Westhorp G, Greenhalgh J, et al. Quality and reporting standards, resources, training materials and information for realist evaluation: the RAMESES II project. Heal Serv Deliv Res. 2017;5(28):1–108.
- Wong G, Greenhalgh T, Westhorp G, et al. Realist methods in medical education research: what are they and what can they contribute? Med Educ. 2012 Jan;46(1):89–96.
- Wong G, Westhorp G, Manzano A, et al. RAMESES II reporting standards for realist evaluations. BMC Med. 2016;14(1). DOI:https://doi.org/10.1186/s12916-016-0643-1
- Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
- Lefroy J, Hawarden A, Gay S, et al. Does formal workplace based assessment add value to informal feedback? MedEdPublish. 2017;6(1):1–15.
- Branch WT, Paranjape A. Feedback and reflection: teaching methods for clinical settings. Acad Med. 2002;77(12):1185–1188.
- Thompson BM, Teal CR, Rogers JC, et al. Ideals, activities, dissonance, and processing: a conceptual model to guide educators efforts to stimulate student reflection. Acad Med. 2010;85(5):902–908.
- Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2009;14(4):595–621.
Appendix A.
Appendix B.
Topic guide for focus groups
1. Provide the students with the statistics for usage of the feedback portal by their cohort of students
Ask: what might explain these figures?
2. In your own experiences what did the email inviting you to respond to your feedback make you do?
3. Did being invited to write about your feedback add any value? If so, what. If not, any reasons?
(did you learn by writing about the feedback? Did you improve clinically or/notice improvements as a result?)
(how do you feel about being asked to make a written reflection?)
4. What have you done with your feedback summaries? And with the learning from feedback+ summaries?
(shown them to anyone? Discussed them with anyone? Put in portfolio?)
5. Assessor gets a copy – is this a help or a hindrance? Who would you like to get a copy?
6. How might the learning from feedback system be improved?
(Timing? Wording? Reminders? Presentation? Access via feedback portal or in e-portfolio? Mandatory? Frequency if so?)