Assessment and feedback to facilitate self-directed learning in clinical practice of Midwifery students

Abstract

Background: Clinical workplaces are hectic and dynamic learning environments, which require students to take charge of their own learning. Competency development during clinical internships is a continuous process that is facilitated and guided by feedback. Limited feedback, lack of supervision and problematic assessment of clinical competencies make the development of learning instruments to support self-directed learning necessary.

Aims: To explore students’ perceptions about a newly introduced integrated feedback and assessment instrument to support self-directed learning in clinical practice. Students collected feedback from clinical supervisors and wrote it on a competency-based format. This feedback was used for self-assessment, which had to be completed before the final assessment.

Methods: Four focus group discussions were conducted with second and last year Midwifery students. Focus groups were audiotaped, transcribed verbatim and analysed in a thematic way using ATLAS.ti for qualitative data analysis.

Results: The analysis of the transcripts suggested that integrating feedback and assessment supports participation and active involvement in learning by collecting, writing, asking, reading and rereading feedback. Under the condition of training and dedicated time, these learning activities stimulate reflection and facilitate the development of strategies for improvement. The integration supports self-assessment and formative assessment but the value for summative assessment is contested. The quality of feedback and empowerment by motivated supervisors are essential to maximise the learning effects.

Conclusions: The integrated Midwifery Assessment and Feedback Instrument is a valuable tool for supporting formative learning and assessment in clinical practice, but its effect on students’ self-directed learning depends on the feedback and support from supervisors.

Introduction

Clinical internships are an essential phase of health professions education during which students develop their competencies in authentic clinical environments (Daelmans et al. 2006; van Hell et al. 2008). According to modern theories of workplace learning, competency development is a continuous process that is facilitated and guided by feedback (Fox & Bennett 1998; Andersen et al. 2008). This article describes a learning instrument designed to support students in taking charge of their own learning in the workplace.

The clinical workplace is a hectic and dynamic learning environment, characterized by high workload and conflicting demands of service and training (Irby & Bowen 2004; White 2007; Ramani & Leinster 2008). This reality requires students to develop new competencies including self-directed learning techniques (Walton & Elliott 2006).

Hammond and Collins (1991) describe self-directed learning as ‘a process in which learners take the initiative, with the support and collaboration of others. For increasing self- and social awareness; critically analyzing and reflecting on their situations; diagnosing their learning needs with specific reference to competencies they have helped identify; formulating socially and personally relevant learning goals; identifying human and material resources for learning, choosing and implementing appropriate learning strategies; and reflecting on and evaluating their learning’.

Although, support and collaboration are prerequisites for effective workplace learning, many studies have reported that supervision can be problematic. Frequently mentioned areas of difficulty are the continuity and frequency of supervision (Schweinfurth 2007; Seifan et al. 2008) and the provision of feedback and support of self-directed learning. Students see direct observation and constructive feedback as key features of effective clinical learning (van der Hem-Stokroos et al. 2003). Feedback can facilitate reflection and self-assessment (White 2007) but, unfortunately, students do not always receive adequate feedback (Branch & Paranjape 2002). Teachers may either neglect to give feedback altogether or the feedback fails to make trainees aware of their strengths and weaknesses in a manner that is conducive to learning. As a result, students are unable to evaluate whether they are achieving their learning goals, developing new goals or making plans to pursue those goals (Ende 1983). Moreover, inadequate feedback does not tell students where they are relative to where they ought to be and where they should go.

Written feedback can be read, reread, archived and exchanged, and thus be a source of information to support self-reflection and authentic assessment. Although research has shown that assessment is a powerful driving force for learning (Swanson et al. 1995; Norcini & McKinley 2007), assessment of clinical competencies remains problematic. Many assessment methods cover only a limited range of competencies and often competencies are not assessed in the context in which they are learned (McKinley et al. 2008). Next to this, assessment criteria are often ill defined, and there is a lack of standardized methods for focussed assessment, and not enough opportunity for reflection, specific feedback and regular monitoring (Borel-Rinkes et al. 2008).

In order to address some of the above-mentioned problems, we developed an instrument that integrates feedback and assessment and is aimed at supporting self-directed learning in the clinical workplace. We conducted a qualitative focus group study to explore students’ perceptions of the instrument.

Our main research questions were

1. What is the effect of continuous and longitudinal written feedback on students’ self-directed learning in clinical practice?

2. What is the effect of the integration of feedback and assessment in self-directed learning in clinical practice?

3. What is the role of supervision in self-directed learning based on the integration of feedback and assessment?

Table

Practice points

Integrating feedback and assessment in a workplace learning instrument supports active involvement in learning by collecting, writing, asking, reading and rereading feedback. Continuous collected written feedback provides information for assessment. A checklist supports self-assessment as a quick scan of the learning progress. The integration is valuable for formative learning and assessment but the value for summative assessment is contested. In order to be effective, students and their supervisors should be motivated and trained. The provision of dedicated time for reflective writing and dialogue should optimize the effect of an integrated learning and assessment instrument. Future research should address the value on summative assessment.

Methods

Context

This study was carried out at the Midwifery Department of University College Arteveldehogeschool Ghent, Belgium. The three-year programme in Midwifery that is offered by the school consists of a modular, competency-based curriculum based on a framework of 24 medical and generic competencies, related to six professional roles. In order to support the development of students’ competencies, the Midwifery department has integrated into the curriculum a programme aimed at enhancing self-directed learning skills. From the first week of this programme, students receive information about the different parts of the programme: the acquisition and assessment of the competencies, the giving and receiving of feedback on the competencies and how to reflect on the competencies.

The Midwifery students have internships in different settings as the maternity ward, delivery ward, gynaecology, neonatology and first-line perinatal care. During the internships, students are guided and supported by a clinical supervisor in the workplace and a teacher from the Midwifery department. Teachers are all midwives with clinical experience. The teacher pays a weekly visit to students in the workplace. Both the clinical supervisor and the teacher take up the educational (supervision of the learning process) and the clinical role (provision of patient care with the student). Normally, teachers are more focussed on the overall learning process and clinical supervisors emphasize the observation during patient care.

As students received limited feedback, we started in 2006 with the development of the Midwifery Assessment and Feedback Instrument (MAFI) in order to support the learning and assessment of the competencies based on principles of self-directed learning. According to the definition of Hammond and Collins of self-directed learning, students were made responsible for their own learning. They had to take the initiative to ask for feedback and to reflect on competencies. By comparing the written feedback with learning outcomes (LO), they were stimulated to diagnose their learning needs and to evaluate their learning (Figure 1).

Figure 1. Format of the learning and assessment instrument.

In MAFI, the 24 competencies and the 6 roles of the Midwifery programme are presented within a framework. In relation to each internship, the relevant competencies in this framework are emphasized. MAFI is a paper and pencil method presenting a format for the feedback unit and the assessment unit. In the feedback unit, there is space for written feedback and written reflections about the selected competencies. Oral feedback can be written by the student, the clinical supervisor, the teacher and any staff member who observes and works with this particular student. It is the students’ responsibility to ensure that sufficient feedback on their progress in all the competencies has been collected at the end of the internship period. Written feedback from students is authenticated by the supervisor by his/her signature. The assessment unit contains a checklist of the selected competencies that students must master during the internship. Each competency results in a specification of a set of LO for the internship. In the checklist, different levels in competency mastery are reflected with a colour code that are next applied in relation to each curriculum year the internship has been set up (green: year 1, red: year 2, blue: year 3). This helps students to indicate whether the LO for that specific year have been accomplished (Pass) or unaccomplished (Fail). In order to help students make this judgement, the LO are defined and expressed in concrete terms. The checklist is used by students for self-monitoring and self-assessment. At an assessment meeting halfway the internship, students and their clinical supervisor and teacher compare the information in the feedback unit with the information in the assessment unit. This assessment is formative, and aimed at improving the student's performance. Although, it is not always possible to realize, a final assessment meeting at the end of the internship with the student, the clinical supervisor and the teacher is recommended to arrive at a pass/fail decision. At the end, the student's performance is graded by a school committee, which judges the student's overall internship performance. Students were informed about the concept and the use of MAFI, and they were trained in writing feedback.

Data collection

A total of four focus groups were conducted: two with second year students (n = 23) and other two with third year students (n = 10). The focus groups comprised a representative sample of Midwifery students with different clerkship experiences on maternity and delivery wards. All the second and last year Midwifery students (n = 108) from the University College Arteveldehogeschool Ghent, using MAFI during those clerkships, were invited by letter to participate in one focus-group session on two selected dates. Participation was voluntary and all participating students signed consent forms prior to the sessions.

The focus group method was chosen because it is an appropriate method to elicit a wide range of ideas and opinions on a well-defined topic.

Procedure

The study was conducted in the summer of 2008. The focus groups lasted 60–90 min, and were facilitated by a member of the research team Erik Driessen) and a researcher of the Midwifery department who was not involved in the training of the students. This was done to ensure that participants felt free to express their views on MAFI without any hierarchical pressure. At the start, the moderators assured the students that full confidentiality was guaranteed.

Discussions were audiotaped and field notes were written immediately following each session. Audiotapes were transcribed ad verbatim by Mieke Embo.

The topics addressed by the focus groups were the value of the integrated feedback and assessment instrument for students’ learning during internships and the specific value of the feedback unit and the assessment unit, respectively. Other topics were the value of MAFI for supervision and the role of the supervisors in facilitating the use of MAFI.

Analysis

The focus group interviews were analysed using the program ATLAS.ti. Analysis was carried out at two levels using accepted qualitative procedures (Miles & Hubermann 1994). Content analysis was performed by two researchers Mieke Embo, Leen Lauwers), who independently coded the data in line with the research questions and categorized the students’ views. They compared their findings and resolved any differences by discussion until consensus of themes and sub-themes was attained. Another researcher reanalysed all the data, using a constant comparative method to establish connections and relationships, and to identify central themes in relation to the research questions. In order to ensure the trustworthiness of the analytic process, we conducted member checking by submitting the results of the analysis and the transcripts to two members of each focus group. Based on their comments more attention was paid to the impact of time issues and to negative feedback.

Results

We present the results for the central themes that were identified in relation to the research questions.

Feedback and self-directed learning

Although, students said that they would appreciate continuous and longitudinal written feedback from supervisors, more than 90% of the feedback from supervisors during the practice period was written by the students themselves. This written feedback was authenticated with a signature from the supervisor who provided the feedback. The teachers wrote feedback, but they came to the workplace only once or twice and the clinical supervisors wrote almost no feedback. According to the students, the clinical supervisors did not write feedback due to lack of time, lack of competence (in respect of what, where and how to write feedback), lack of motivation and reluctance to give feedback, in relation to interpersonal competencies in particular.

All students thought that the writing of feedback in MAFI stimulated reflection. One participant said

‘It takes a lot of time but I do find that because I have to write it down, I have to think about what went well and what did not go well and why’.

Although the supervisors rarely provided written feedback, they did give verbal feedback. Students were motivated to ask for it because it was instructed that every competence on MAFI needed evidence before entering assessment. Students reported that verbal feedback from supervisors facilitated their reflective writing about competencies. Conversely, reflective writing was hampered when there was little feedback, an imbalance of positive and negative feedback or feedback on a limited range of competencies only. Students experienced that feedback did often focus on their weaknesses and that this type of feedback rarely provided guidance for improvement. Some students said that verbal feedback that was predominantly or exclusively negative undermined their confidence, which in turn had a negative impact on their reflective writing. Importantly, all the students reported that the feedback did not cover the full scope of the competencies on MAFI, with teachers and clinical supervisors tending to limit their feedback to medical-technical competencies to the neglect of general competencies.

Students said that they had to be motivated to produce daily written feedback. Motivation appeared to depend on different factors. While third year students were more internally motivated by individual growth and personal development, the majority of the second year students were motivated by external factors as there were assessment and supervision.

‘The information can help you to avoid making the same mistakes again or to improve during your next internship’ (third year).

‘The paperwork is a real burden of the internship. The only benefit is for the assessors, so they can read if you have done it right’ (second year).

Due of the continuous and longitudinal training of writing feedback, students made progress in their writing. Although writing feedback was difficult, third year students had developed strategies that made it easier. They wrote feedback immediately after working with, or without, a clinical supervisor, asking the clinical supervisor to supplement or correct student's written feedback. Second year students strongly depended on external input for feedback. They were afraid to ask for more feedback than they received, especially on generic competencies. They were afraid of receiving negative feedback, which could undermine their self-confidence. However, at the same time they were aware that it was their responsibility to ask for feedback.

The amount of needed time for writing feedback was related to the way the MAFI was structured. Most students were positive about the effects of reflective writing but preferred a less structured and detailed instrument so that writing would take less time and could be discussed with the clinical supervisor.

Continuous and longitudinal written feedback enabled students, teachers and supervisors to read and reread the feedback given during the learning period. Students and teachers did read the feedback but clinical supervisors did so very rarely. There was a general agreement among the students that rereading feedback was valuable because it reminded them of advice and helped them to develop strategies for improvement.

‘I read feedback in the evening or in the morning or just before practical work so that it is fresh in my mind and then it helps me to learn. There are always things that you remember, but others that you forget’.

However, students also said that they did not reread unexpected negative feedback, because it undermined their self-confidence.

Teachers were motivated to read the feedback because they hardly ever observed students, and therefore needed the information for the final assessment.

The clinical supervisors hardly paid any attention to the continuous collected written feedback. Students reported that clinical supervisors did not have enough time to do so and failed to recognize that substantial benefits might be gained from observing students’ performance.

Assessment and self-directed learning

All the students were motivated to use the assessment checklist and thought the checklist was valuable because it provided a concrete and easy-to-use overview of their LO. The amount of detail was not considered a barrier. Quite the opposite, students reported that the details made the checklist useful as a ‘quick scan’ of their learning progress.

The third year students said the written feedback was relevant in view of self-assessment. Students replied to the question whether the instrument would have the same effect without the written feedback as followed:

‘The checklist is the conclusion, but it doesn’t tell you how well you are doing. If you write feedback you have more information’.

‘The feedback unit is very broad and with the checklist, you see immediately where you have to work on’.

Being stimulated to take responsibility for their own learning by reflecting on LO and competencies appeared to be of crucial importance to students. Interestingly, they mentioned different activities, as described in the definition by Hammond and Collins (1991): (a) monitoring competencies, (b) setting goals, (c) diagnosing gaps and learning needs and (d) asking for learning opportunities. These activities are illustrated by the following quotes.

‘… because you put yourself under a magnifying glass. You assess yourself. Yes that is insightful. You can learn a lot from your own actions, such as the initiatives you take. When you have to reflect on this, you can grow by looking back; you pause and then decide what you take forward and what you leave behind’ (a–b).

‘I do look at my checklist, for example when I am in the middle of an assignment, and then I tick what I have done and what I still have to do … and also, at the start of the placement, which outcomes I am going to achieve …’ (a–c).

‘It's also my responsibility to remind the midwives of the type of learning opportunities I want to have’ (d).

The integration of feedback and assessment was perceived as valuable for formative learning and assessment, but its value for summative assessment and grading was contested. Despite the efforts of the designers of MAFI to separate teaching and assessment, as is recommended in the literature, students appeared to attach much importance to the involvement of clinical supervisors in their assessment. Although during their day-to-day activities the students were mostly supervised by them, their final assessment was often determined in a discussion between the student and the teacher. Students felt uncomfortable that their clinical supervisors did not always contribute to the final assessment. Third year students argued in support of a final assessment meeting in which the student, the clinical supervisor and the teacher were present. They contended that in this way the written feedback could be controlled and optimized and that such a dialogue allowed comparisons between the judgements of the student, the clinical supervisor and the teacher. As one student said:

‘I thought it was very useful because you have to think for yourself: What have I achieved? But you also get confirmation from your clinical supervisor and the teacher that ‘Yes, I agree you did that’ or you may think ‘Yes I achieved that goal’, but the clinical supervisor says ‘some further work remains to be done’. The fact that someone says that is very important’.

Supervision and self-directed learning

MAFI is a learning instrument based on providing feedback, assessment and support. Students described the role of the clinical supervisors and some conditions they thought would support self-directed learning with MAFI. They focussed on the clinical supervisors, who to them were most important with regard to the effect of MAFI.

‘Empowering students to engage in self-directed learning with MAFI’ and ‘providing feedback that could be written in MAFI’ are phrases that summarize what students expect from their clinical supervisors. Actions of the clinical supervisors that empowered students to regulate their learning were considered to boost students’ confidence in self-directed learning activities as there are asking, writing and reading feedback, diagnosing learning needs with the checklist and asking for learning opportunities, reflecting on new performances and competency development, a good ‘student clinical supervisor relationship’ was prerequisite for building self-confidence. For second year students, the quality of the student–supervisor relationship was related to ‘a positive feeling about working together’ and for third year students it was related to their feeling of being empowered to self-direct their learning process. As one third year student puts it

‘I think that it's essential for the clinical supervisor to be concerned and involved. If that is the case, the supervisor will take an interest in your overall learning process and give you the opportunity to improve in a good and comforting way’.

The student–supervisor relationship evoked strong emotions from the students. A positive impact on self-confidence was related to a sense of success, responsibility and encouragement. Negative effects on self-confidence led to stress, depression, feelings of inferiority, fear and a sense of unfairness. Students felt that they received more feedback from supervisors who were motivated to guide them. While some students talked about clinical supervisors who ‘loved to teach’, others referred to clinical supervisors who were clearly unwilling to undertake the teaching role.

Students’ acceptance of feedback was also linked to the quality of the student–supervisor relationship. Within a good relationship the continuous and longitudinal collected written feedback was used formatively and focussed on the development of competencies by comparing the feedback with the checklist. As we saw earlier, second year students mostly focussed on assessment and third year students on the progress they made in the development of competencies. If the student–supervisor relationship was sub-optimal, third year students too focussed mainly on summative assessment. This was particularly relevant if the feedback was negative.

Supervising the self-directed learning of students in the workplace with MAFI requires specific competencies. The students said they often encountered a lack of teaching competencies in their clinical supervisors. They suggested staff development activities to improve supervisors’ and teachers’ competencies in relation to: how to supervise students, how to give and write feedback, how to support the development of competencies and how to determine, which competencies should be trained and assessed.

According to the students, staff development activities should tackle the use of ‘MAFI,’ because the instrument was underused and most clinical supervisors did not know how to use it properly.

‘They don’t understand the relevance of the two units. They see it more as a list of activities than as a feedback instrument’.

Discussion

We explored students’ perceptions of the effects of an integrated instrument for feedback and assessment during internships in Midwifery practice. Students generally agreed that the integration of feedback and assessment supported self-directed learning, provided they received feedback from motivated and competent clinical supervisors. They also appreciated that MAFI made it possible to have an active role in their own development. The instruction they were given to collect written feedback about all the competencies by writing feedback, asking for feedback and reading and rereading feedback was experienced as time consuming and easier said than done. But, it was also considered to promote self-reflection, self-monitoring and the creation of personal action plans by asking ‘what went well, what should be improved and how (Bienstock et al. 2007)’. The assessment unit of MAFI was seen as a ‘quick scan’ of LO, which could be compared with evidence from the feedback unit and thus enabled monitoring and formative assessment of competencies. The assessment unit was considered to support self-assessment. However, the students doubted the value of the integration of feedback and assessment for summative assessment and regretted that the possibilities for supervisors to support their self-directed learning were generally underused.

The students reported that continuous and longitudinal feedback on self-directed learning enhanced their motivation to take the initiative in writing feedback on all the competencies. We saw a shift in motivation concerning the written feedback between the second and the third year. Where second year students were mainly externally motivated (by assessment and supervisors), third year students were more internally motivated to use feedback to diagnose learning needs and develop plans for improvement. Growing confidence in the learning process and development of competencies might optimize the effect of MAFI. In line with results reported in the literature, the students identified effective feedback as a key factor in self-directed learning (Heron 2008; Koh 2008). The role of the supervisor was more important than the role of the instrument itself and students stated that clinical supervisors and teachers should increase their efforts to provide effective feedback. While the use of clinical performance ratings is not undisputed in the literature (Govaerts et al. 2007), all the students in this study appreciated the overview of the competencies and LO that was offered by MAFI (van der Vleuten & Schuwirth 2005; Harden 2007). This overview was seen as a quick scan for students. Feedback can be a part of assessment and other studies have shown that this motivates students to take responsibility to monitor their own learning, reflect on competency growth and look for learning opportunities (Bienstock et al. 2007; Koh 2008). In this study, the students constantly compared their own performance with standards in the checklist and this enabled them to identify areas that required further work. The MAFI was valuable for formative learning and assessment but was not perceived as contributive to the quality of the summative assessment. Students were convinced that a final assessment conversation between the student, the clinical supervisor and the teacher was essential and might contribute to the quality of assessment. We need to more strictly adhere to plan to do this final assessment conversation and we foresee that it has the intended effect in further research.

The effect of MAFI on students’ self-directed learning and self-confidence in their own learning process depended on the support from motivated, empathic and competent supervisors. First of all, the supervisors were responsible for providing effective feedback, which could be written by students. As described in the literature, it was generally difficult for the students to collect feedback, on general competencies in particular (Heron 2008). Furthermore, although a balance between positive and negative feedback is generally recommended (Salerno et al. 2003), the students indicated that they suffered when they received negative feedback regularly, saying it undermined their self-confidence with a negative impact on reflective writing. In the literature we found different opinions on this topic. On the one hand, teachers are reported to be very hesitant to provide negative feedback (Ramani & Leinster 2008), but on the other hand there are reports that the overall prevalence of belittlement and humiliation is surprisingly high in the clinical setting (Mattick & Knight 2009). Second, students expected practical and emotional support from their clinical supervisors. The effect of a supporting relationship on learning was often linked with a positive feeling about ‘working together’, but this was more important for second than for third year students. When the student–supervisor relationship was suboptimal, third year students too were more occupied with summative assessment than with learning.

Time or rather time constraints for working with the MAFI were mentioned in different ways during this study. Use of MAFI by supervisors was affected by lack of time. Clinical supervisors did not have sufficient time to give feedback and hardly wrote feedback, although students very much appreciated it when they did so. Students suggested that the feedback unit might be changed to become less detailed in order to make the clinical supervisors more motivated to write. There was also a positive time issue: when MAFI is used students and supervisors are compelled to devote more time to the learning process and to supervision. This issue is actually being dealt with in a follow up study investigating the perceptions of supervisors of the value of MAFI on the support of self-directed learning in clinical practice.

Several limitations of this study need to be considered. The most important limitation was the self-reported nature about the effects of MAFI on the learning of the students. Another limitation emphasizes that the participating students were volunteers, which may have biased students’ responses. Furthermore, the participating group represented only a small percentage of the entire student cohorts. Because of the small sample and specific context of Midwifery education in Belgium, the generalizability of the results may be limited.

Conclusion

The results of this study suggest that the integration of feedback and assessment in a clearly defined learning and assessment instrument is a potentially valuable method to promote self-directed learning and formative assessment during internships. However, the students contested the instrument's value for summative assessment, at least in this current form, and the instrument appeared to be undervalued and underused by supervisors. Feedback and motivated, competent supervisor(s) are essential for successful effect of MAFI on self-directed learning in practice. Those intending to use an integrated instrument, such as MAFI should pay attention to the training of students, clinical supervisors and teachers in the use of the instrument and provide dedicated time for reflective writing and dialogue. Provided it is used as intended, an instrument for the integration of feedback and assessment in an authentic clinical setting can provide opportunities for supporting self-directed learning in the workplace.

Acknowledgements

We thank the students of the Midwifery department of the University College Arteveldehogeschool Ghent for their participation in this study. We also thank Leen Lauwers for her assistance with the interviews and Mereke Gorsira for editing the final version of this manuscript.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.