Students’ reflections in a portfolio pilot: Highlighting professional issues

Abstract

Background: Portfolios are highlighted as potential assessment tools for professional competence. Although students’ self-reflections are considered to be central in the portfolio, the content of reflections in practice-based portfolios is seldom analysed.

Aim: To investigate whether students’ reflections include sufficient dimensions of professional competence, notwithstanding a standardized portfolio format, and to evaluate students’ satisfaction with the portfolio.

Methods: Thirty-five voluntary final-year medical students piloted a standardized portfolio in a general practice (GP) attachment at Lund University, Sweden. Students’ portfolio reflections were based upon documentary evidence from practice, and aimed to demonstrate students’ learning. The reflections were qualitatively analysed, using a framework approach. Students’ evaluations of the portfolio were subjected to quantitative and qualitative analysis.

Results: Among professional issues, an integration of cognitive, affective and practical dimensions in clinical practice was provided by students’ reflections. The findings suggested an emphasis on affective issues, particularly on self-awareness of feelings, attitudes and concerns. In addition, ethical problems, clinical reasoning strategies and future communication skills training were subjects of several reflective commentaries. Students’ reflections on their consultation skills demonstrated their endeavour to achieve structure in the medical interview by negotiation of an agenda for the consultation, keeping the interview on track, and using internal summarizing. The importance of active listening and exploration of patient's perspective was also emphasized. In students’ case summaries, illustrating characteristic attributes of GP, the dominating theme was ‘patient-centred care’, including the patient–doctor relationship, holistic modelling and longitudinal continuity. Students were satisfied with the portfolio, but improved instructions were needed.

Conclusions: A standardized portfolio in a defined course with a limited timeframe provided ample opportunities for reflections on professional issues. Support by mentors and a final examiner interview contributed to the success of the portfolio with students. The interview also allowed students to deepen their reflections and to receive feedback.

Introduction

Over the past 10–15 years, a variety of portfolios have been introduced as learning and assessment tools in undergraduate medical education (Driessen et al. 2007; Buckley et al. 2009). The main reason for this development has been a shift of focus in medical school curricula from acquisition of knowledge to development of competence.

Professional competence in medicine involves not only the use of knowledge and skills, but also integrates attitudes, values and an aptitude for communication, clinical reasoning and self-reflection (Epstein & Hundert 2002), attributes not easily assessed by traditional methods. The portfolio has been highlighted as a potential tool for learning and assessment of professionalism, as it emphasizes the development of reflective practice (van Tartwijk & Driessen 2009; Passi et al. 2010). A portfolio from clinical practice holds students’ documentary evidence from the workplace, combined with a self-reflection (Snadden & Thomas 1998; Friedman Ben David et al. 2001).

Reflective ability is identified as a core skill for professional development (Friedman Ben David et al. 2001). Several definitions of the term reflection exist (Mann et al. 2007; Sandars 2009). Sandars proposes a rather wide definition: ‘a metacognitive process that occurs before, during and after situations with the purpose of developing greater understanding of both the self and the situation so that future encounters with the situation are informed from previous encounters’. Reflection is also an important stage in Kolb's well-known ‘experiential learning cycle’, where an experience in practice is translated through reflection into abstract generalized concepts that can be applied in new situations (Kolb 1984). A reflection for learning in a portfolio is supposed to include analysis of educational achievements, identification of further learning needs and a learning plan (Challis 1999). Besides reflection for learning, reflection is also essential for building therapeutic relationships, a crucial part of professional competence (Epstein & Hundert 2002; Sandars 2009).

Reflections in a portfolio can be assessed separately from the documentary evidence (Friedman Ben David et al. 2001). Several instruments for the judgement of reflective writing exist (Mann et al. 2007; Sandars 2009), and are usually based on different ‘levels’ or ‘depths’ of reflection. The most superficial level means only a description of an event; the deepest level involves an analysis and an application of the reflective learning to future experiences.

Assessment of the portfolio is essential to foster reflective learning through feedback and to motivate and support further learning (Challis 1999). Summative assessment emphasizes the value of the portfolio to students, and has been highlighted as one of the key factors for a successful portfolio (Driessen et al. 2007). Inter-rater reliability can be improved by standardization of portfolio content, definition of analytical assessment criteria, clear guidelines to students (Friedman Ben David et al. 2001) and discussion and negotiation between examiners (Rees & Sheard 2004). Reliability can also be enhanced by a personal examiner interview (Davis et al. 2001; Gordon 2003; Burch & Seggie 2008).

Nevertheless, there is a crucial balance between standardizing the portfolio to enhance reliability and maintaining the personalized aspect of the portfolio (Driessen et al. 2005). Because portfolio assessment is primarily concerned with qualitative material, an alternative approach of using qualitative research criteria in assessment has been suggested (Webb et al. 2003; Driessen et al. 2005). In this approach, ‘credibility’ (internal validity) is used for the assessment structure and ‘dependability’ (reliability) for the assessment process.

The use of portfolios in undergraduate medical education has been reported in several papers. In pre-clinical education portfolios have been introduced to stimulate students to reflect on professionalism (Driessen et al. 2003; Gordon 2003) or their communication skills (Rees & Sheard 2004). In clinical practice, they have mainly been used in lengthy clinical attachments (Finlay et al. 1998; Davis et al. 2001; Burch & Seggie 2008), but they have also been applied to defined courses of shorter duration (Lonka et al. 2001; Amsellem-Quazana et al. 2006; Grant et al. 2007). However, although students’ personal reflections are considered to be a central attribute in the portfolio, reflections were not included in all of these portfolios from clinical practice, and if reflections were included, the content was seldom subjected to analysis. Also, only a few researched portfolios from clinical practice used reflections to promote the integration of affective and cognitive dimensions of professional competence.

We aimed to develop a portfolio for the assessment of final-year students’ professional development during their general practice (GP) attachment. Considering the limited time available for students to collect portfolio evidence and to write reflections, the portfolio had to be structured and standardized. We designed a pilot study with the principal aim of investigating whether themes of students’ reflections would include satisfactory dimensions of professional competence, notwithstanding the standardized format. This article qualitatively explores the emergent themes of students’ reflections, and also reports on students’ evaluation of the portfolio.

Table

Practice points

Students’ reflections in the learning and assessment portfolio in GP include sufficient professional issues of cognitive, affective and practical dimensions, notwithstanding a standardized format and a limited timeframe. Mentors can increase the acceptance of the portfolio with students. A final examiner interview is recommended to deepen students’ reflections and to supply feedback.

Methods

Context

The medical school curriculum of Lund University, Sweden, has an intake of approximately 80 students twice a year, comprises five and a half years, and is followed by an internship of 18 months before qualification. During the revision of the curriculum in 2006, the demand for additional methods for the assessment of professional development was emphasized. The portfolio seemed to be a tool particularly well-suited for our final-year students, who were in a phase of medical training characterized by a large degree of authentic learning in the workplace. We aimed to use the portfolio for formative promotion of learning, but also as a complement in the summative assessment.

The programme of Community Medicine runs for 10 weeks in students’ final year, and includes 16 days (4 days every 2nd week) of practice in a health centre. The final exam is a written case-based test on GP and other course subjects.

Portfolio

Box 1 outlines the learning outcomes, the content and the areas for assessment of the portfolio. Assessment protocols (Box 1) are used in practice for self-assessment by students and for formative assessment by their GP supervisor. Students also utilize an 8-item checklist of practical skills (Box 1), and record patient consultations on video. The recordings are discussed in group seminars, where students’ personal reflections on their consultations are completed by feedback from their peers and teacher on a feedback sheet.

Box 1 The portfolio: learning outcomes, content and areas for assessment

Learning outcomes

The student is expected to

• analyse his/her strengths and weaknesses in GP consultations, reflect on learning achieved, what areas need further development and how this development may take place

• discuss and critically examine characteristic attributes of GP, illustrated by two case summaries from practice, reflect on learning achieved and on how further development may take place

Content

Documentary evidence

• Assessment protocols (self-assessment and feedback) from practice (communication skills; history taking; physical examination; differential diagnoses; problem solving; management plan; explanation with patient and relationship/efficient use of time)

• Checklist of practical skills from practice (provide lifestyle advice to patients; examine patients with neck/shoulder pain or low lumbar pain; consult with old patients with multiple complaints; document patients’ records; write hospital referral letters; write sick-notes; examine children in child welfare clinic and accompany the GP on home visits)

• Feedback sheet from video seminar

• Two case summaries (each 250–500 words), critical incidents or routine clinical experiences

Reflections

• One 500-word reflection on consultation skills, linked to documentation from practice and video group

• Two 250–500-word reflections, one on each case summary, linked to one or more of the 11 characteristic attributes of GP (WONCA Europe 2005)

Areas for assessment

in each of the three reflections

• Reflection on learning achieved, further learning needs and a learning plan

• Use of documentary evidence/GP attributes

in the whole portfolio

• Structure and written language

The portfolio content was standardized to hold documentary evidence from the practice and the video seminar group, and also included two case summaries, intended to illustrate one or more of the 11 characteristic GP attributes (WONCA Europe 2005) that were supplemented (Box 2).

Box 2 Characteristic attributes of GP (WONCA Europe 2005)

Accessibility – first contact for all health care problems

Co-ordinating care – with other professionals in primary care and with other specialities

Patient-centred approach – centred on patient and context

Patient–doctor relationship

Longitudinal continuity of care

Specific decision-making process – based on incidence and prevalence

Non-specific undifferentiated symptoms

Simultaneously manage acute and chronic health problems in individual patients

Promote health and well-being – prevention

Community orientation – responsible for the health of the community

Holistic modelling – physical, psychological, social, cultural and existential dimensions

Essential application features – attitudinal, scientific and contextual aspects – are important for all doctors to apply the above core competencies in real life in the workplace setting.

Three reflections on learning were required: one reflection on consultation skills, based upon documentation from the practice and video group, and two reflections on the two case summaries, where students’ learning could deal with anything about the patient encounter and/or with the attributes (Box 2).

Pilot

Students of two consecutive classes, of 76 and 71 students, respectively, were approached. Twenty students in each class were offered to voluntarily pilot a portfolio in GP as an addition to the written test, which for the participants would be reduced in the number of GP questions. A total of 35 students, 16 from the first and 19 from the second class, accepted. Twenty-four (69%) of them were women, as compared to 59% of all students. At the time of the study, consent from the University's ethics committee was not required for projects involving students’ assessments. Participating students were provided with both verbal and written guidelines on the purpose, format and assessment of their portfolios. Three of the authors (ACH, AP, AB) and a fourth teacher, all involved in teaching, acted as mentors to support their students in compiling the portfolio.

All documentary evidence, except for the case summaries, was formatively assessed in practice or in seminars, and the summative assessment only concerned the three reflections. Assessment in medical school, Lund University, is non-graded, pass/fail. The assessment of each of the three reflections was based upon three areas: the reflection of learning, the use of documentary evidence/GP attributes, and the structure and written language of the whole portfolio (Box 1). The four teachers acted as ‘external’ examiners for each other's students. Each portfolio took 1 hour to read and to prepare for the 30-minute personal interview with the student. The evaluation of ‘borderline’ students’ reflections was discussed between examiners. The assessment was holistic, and the interview provided students with an opportunity to deepen their reflections.

Qualitative analysis of students’ reflections

To be able to conduct the qualitative analysis, students’ reflections were written verbatim into a word processor by one of the authors (ACH). Each student was allocated a code number to prevent identification of individual students’ comments. The text of the reflections constructed two separate units of analysis; ‘consultation skills’ and ‘case summaries’.

The transcripts were read through several times to get a sense of the whole, and then analysed using a framework approach (Crabtree & Miller 1999). For ‘consultation skills’, the text was mapped against the Calgary Cambridge Guides (Silverman et al. 2004). For ‘case summaries’, the statement of WONCA Europe (2005), (Box 2), was used as a template for organizing the data.

In each of the two units of analysis, related segments of text were identified and indexed to the appropriate part of the thematic framework. Some codes were modified and new ones added to reflect as many nuances in the data as possible. ‘Chunks’ of related text under each code were examined together and distilled summaries made; this process involved a considerable amount of abstraction. The chunk summaries provided an analytical tool from which connections were made and sub-themes and themes identified.

ACH undertook the mapping of text and carried out the primary analysis of the data. The four teachers, involved in the pilot, examined in recurrent meetings whether the interpretation of the data was plausible. The codes and sub-themes were compared based on differences and similarities, and some were revised during this process of corroboration.

Students’ evaluation questionnaire

The short student evaluation form was supplied directly after the examiner interview, asking five questions about the portfolio instructions, the mentor contact and the examiner interview, using a 3-point scale (‘yes’, ‘to some extent’, ‘no’). Students were also requested to provide free comments on positive and negative experiences of the portfolio, and to give suggestions for improvements. The results were subjected to quantitative and qualitative (content) analyses.

Results

Themes of students’ reflections on their consultation skills

The analysis resulted in the assignment of 26 codes (Table 1). The codes were merged into 11 sub-themes. From the sub-themes, four themes were identified: ‘Process skills’, ‘Perceptual skills’, ‘Content skills’ and ‘Context’. The first three are congruent with the three broad types of skills needed for the medical interview (Silverman et al. 2004), and the fourth theme ‘Context’ was added as a supplement.

Table 1.  Students’ reflections on their consultation skills (number of comments in brackets)

Codes	Sub-themes	Themes
Time management	Structure (51)	Process skills (103)
Negotiate agenda
Keep interview on task
Summarize periodically
Listen attentively to the patient's opening statement, without interrupting or directing patient's response	Active listening (17)
Moving from open to closed questions
Facilitate patient's response by the use of silence
Actively determine and explore patients’ ideas, concerns and expectations	Patient's perspective (13)
Pick up verbal and non-verbal cues
Use empathy to communicate understanding of patient's feelings
Write notes without interference with dialogue or rapport	Building relationship (11)
Deal sensitively with embarrassing and disturbing topics
Use concise, easily understood language Provide clear information Relate explanation to patient's perspective	Reaching shared understanding (11)
Self-concept and self-confidence Awareness of one's own feelings Awareness of one's own issues of concern	Self-awareness (20)	Perceptual skills (41)
Internal clinical reasoning	Clinical reasoning (11)
Future communication skills training	Communication skills (10)
Ask relevant questions to explore patient's problems Formulate management plan Maintain continued learning	Scientific aspects (21)	Content (34)
Elicit and interpret signs in physical examination Future practical skills training	Practical skills (13)
Co-ordinated care	Primary care (4)	Context (4)

Process skills

Most comments were assigned to the structure of the consultation. Many students described ‘time management’ as their greatest problem at the start of their practice. In the medical interview, they seemed to ‘get lost’ in details not relevant for patient's immediate problems. They learnt how to improve structure mainly by using three different skills: negotiating an agenda for the consultation, keeping the interview on track, and frequently using internal summarizing.

Setting an agenda required taking both the patient's and the doctor's needs into account, problems with time could be discussed, and the patient could be invited to take responsibility in proposing the most important problem.

In the negotiation about what the consultation should deal with, I could have given the patient more responsibility in letting him propose the most important problem (student 12)

The interview could be kept on track by obtaining order and control between various sections to be covered. Students learned that they could interrupt over-talkative patients in ‘a nice way’ by acknowledging their comments and gently asking them to refocus, steering them back to the problem at hand.

Now I am aware that I have to steer the interview a bit more and take control, by refocusing on the subject, clearly interrupting, or offering the patient alternatives, depending on the situation (student 27)

Besides playing a vital role in structuring the medical interview, internal summarizing had other advantages. It allowed the student to check the accuracy of the patient's history, it invited the patient to either confirm or correct the student's interpretation, and it encouraged the patient to continue his or her story. It could also be used together with orienting statements to move from one section to another in the interview.

The summary can also be used for ‘checking’ in the middle of an interview, for confirmation of what has been said, and to conclude one section and pass over to the next (student 19)

The importance of active listening, especially at the start of the interview, was realized by several students. By listening attentively without interruption of the patient's opening statement, they could get more information, improve the structure of the consultation, and save time.

At the start of my attachment, I began my questioning as soon as there was a moment's silence, but I learned to let patients decide when they had completed their opening statement. I realized it didn’t take more time and I got to know more (student 32)

To determine the patient's perspective, students learned to ask direct questions about the patient's ideas, concerns and expectations, and to pick up on the patient's cues. The importance of phrasing the questions carefully and using them with ‘timing’ was acknowledged. Improved efficiency of the consultation, when the patient's perspective was made clear, was recognized.

The most useful skill has been to ask the patient for her ideas, concerns and expectations. I soon noticed that if I asked what the patient thought caused the symptoms, what worried her, and what she expected from me, the consultation became much more efficient (student 15)

In building a relationship throughout the consultation, some students had realized the importance of communicating their understanding back to the patient. Others commented on their note-taking during the initial part of the interview, which could deteriorate the rapport with the patient. A few students described how they had finally come to feel at ease asking questions about embarrassing and disturbing subjects, which they considered to be ‘evidence of their professional development’. Others admitted that they still ‘forgot’ to ask these questions, realizing a ‘subconscious fear of the answers’.

In the sub-theme of reaching shared understanding, some students commented on their improved skills of supplying information for patients in a clear and concise way, avoiding medical jargon. Others reflected on shared decision making, ‘placing patients’ own resources at the centre’.

Perceptual skills

Self-awareness was the predominant sub-theme. Several students described improved confidence in their work, partly due to the wide range of patients’ problems experienced, partly due to self-reflection, based on their own analysis of their strengths and weaknesses and on their GP tutor's constructive feedback.

I have improved in receiving and analysing feedback, using the feedback as a constructive tool in my personal development, and I now continuously reflect on my work (student 23)

Some students reflected on the importance of being observant of their own feelings of frustration or anger, as they might be projected from the patient. Others considered a heightened awareness of external factors that could negatively affect the quality of their consultations. By identifying these factors in advance, they could handle the situation with improved confidence.

Clinical reasoning was acknowledged as a problem by several students. They perceived the retrieval of basic and clinical science and the transfer from theory to practice across different problems as a challenge. They had, however, been supplied with constructive criticism and tools for further training by their GP tutors.

My ability to transfer my knowledge from theory to practice hasn’t really been tested until this semester (student 14)

Some students reflected upon their tendency for premature closure on a particular diagnosis, instead of keeping a degree of scepticism and a broader perspective. Others described a tendency to direct their initial reasoning towards the most serious and unlikely of diagnoses, and still others were uncertain of their knowledge base, which made it difficult to generate probable hypotheses.

A third of students reflected on their need for further development of their communication skills, using continuous self-reflection, video recordings of consultations, sit-ins by experienced colleagues, and joint group discussions.

Content skills

Eliciting and interpreting signs in physical examinations were skills practised frequently by students during their previous hospital clerkships. These issues were, however, still commented upon as learning issues by some students, who had considerably increased their ‘reference library’ of normal and pathological findings.

Formulating a management plan was a challenge to most students at the start of their attachment. They recognized the need of a good knowledge base as a background, but lacked the experience to use it across the non-specific complaints presented in GP. Students were encouraged, and sometimes ‘forced’, by their GP supervisors to make suggestions for investigations, treatments or follow-up arrangements for their patients.

By direct feedback from my GP supervisor I have learnt to handle ‘easy’ cases independently, and I have improved in suggesting investigations and treatment in more complicated cases (student 23)

Several students acknowledged the importance of continuously updating their scientific knowledge and practical skills to maintain high quality in their work. This concept included continuous self-evaluation and the courage to seek advice when necessary.

Themes of students’ reflections on their case summaries

Twenty-eight codes were assigned (Table 2), of which 20 were merged into the 11 sub-themes of characteristic GP attributes (WONCA Europe 2005). These sub-themes were condensed into three themes: ‘patient-centred care’; ‘primary care management’ and ‘specific approach to patients’ problems’. The remaining nine codes were merged into another five sub-themes and then condensed into three themes: ‘perceptual skills’, ‘content’ and ‘context’, congruent with the essential application features (Box 2).

Table 2.  Students’ reflections on their case summaries (number of comments in brackets)

Codes	Sub-themes	Themes
Reaching shared understanding Sharing power and responsibility Personal relationship and trust	Patient–doctor relationship (36)	Patient-centred care (89)
Physical, psychological, social, cultural and existential dimensions	Holistic modelling (21)
Oriented to the individual Patient's belief, concerns, expectations, feelings and needs	Patient-centred approach (18)
Longitudinal continuity of care	Longitudinal continuity (14)
First medical contact – open and unlimited access Full range of health conditions	Accessibility (13)	Primary care management (35)
Co-ordinate with other professionals in the primary care setting	Co-ordinating care (15)
Interface with other specialities
In balance with available resources	Community orientation (7)
Disease in early stage, non-specific signs Use time as a tool, tolerate uncertainty Effective and efficient use of diagnostic and therapeutic interventions	Non-specific undifferentiated symptoms (12)	Specific approach to patients’ problems (24)
Decision making based on incidence and prevalence Decision making based on knowledge of patients and community	Specific decision-making process (7)
Hierarchical management of problems	Simultaneously manage acute and chronic health problems (3)
Prevention	Promote health and well-being (2)
Awareness of one's own feelings Awareness of one's own issues of concern Awareness of one's own attitudes	Self-awareness (24)	Perceptual skills (37)
Identifying ethical aspects of clinical practice	Ethics (6)
Future communication skills training	Communication skills (7)
Recapitulate basic and clinical science Read and assess medical literature Maintain continued learning	Scientific aspects (24)	Content (24)
Impact of the community	Contextual aspects (3)	Context (3)

Patient-centred care

This was the dominant overall theme, and the most frequent reflection was on the patient–doctor relationship. Most students commented on the importance of providing adequate, effective information and explanation to patients, encouraging patients to voice their ideas and preferences, with the aim of reaching consensus through negotiation. This was not always possible, however. Several students discussed the patient's responsibility and the doctor's respect for the patient's autonomy, when the patient refused to accept suggested advice, investigations or treatment for his or her condition. Students recognized that power and responsibility are shared, and that an essential task is to supply the patient with information, support and empathy.

I experienced how difficult it might be to try to motivate and help a patient to change his lifestyle and make him understand the seriousness of his condition (student 26)

Patients’ personal relationship with their GP, based on mutual trust and established over time, was acknowledged to be of a much greater importance than students had previously believed.

Holistic modelling in patient contact was an issue in the reflections of several students who had observed underlying psychosocial problems in many of their patients. The exclusion of biomedical disease and reassurance that there was ‘nothing wrong’ was not always enough. The importance of identifying social and psychological factors, of ‘thinking behind the biomedical perspective’ was a valuable experience for some students.

I learnt from this attachment that I have previously missed this holistic approach to patients, even if I have thought a lot about it before (student 14)

Many students emphasized the patient-centred approach, including the understanding of the individual's experience of illness and the importance of eliciting particularly the patient's feelings and fears, and being sensitive to patients’ cues.

Students perceived longitudinal continuity as a unique strength of primary care. Longitudinal continuity was important to avoid unnecessary investigations, to create mutual trust and a holistic approach to patient's problems, and to give adequate advice about the patient's lifestyle, based on the GP's long-standing knowledge of the patient's life settings.

Primary care management

This theme evoked comments mainly on accessibility and co-ordinated care. GP was in most cases perceived as ‘first medical contact’ within the health care system, and students learnt that health problems in GP can cover a wide range from trivial to life-threatening conditions.

Patients know that they can consult with all kinds of problems at the health centre and that they will be referred to a specialist if needed (student 34)

The co-ordinated care with other health professionals in primary care was appreciated by half of the students, who had acquired enhanced respect for the competence of nurses and physiotherapists. Students emphasized the importance of ‘working together as a team’ for the benefit of the patient's quality of care.

Specific approach to the patient's problems

Only one of the sub-themes was assigned a reasonable amount of reflective comments: non-specific undifferentiated symptoms. Taking care of patients with non-specific symptoms, often in the early stage of an illness, made several students feel frustrated and insufficient. They learnt the feasible strategy of using time as a tool, keeping in touch with the patient and awaiting further developments.

Perceptual skills

Self-awareness was the key emergent sub-theme, and the results of the analysis partly overlapped students’ reflections on self-awareness in their reflections on consultation skills. Several students defined difficult consultations, where they had to disregard their own feelings and act professionally. Some reflections indicated concern about the problems in trustworthiness that they might later experience as young doctors. A third of the students reflected on their attitudes towards certain patients, being aware of how their own and their colleagues’ attitudes can influence the start of a consultation. However, several patients did not at all correspond to their prejudiced expectations, and students learned to approach their patients unconditionally.

This was my absolutely most important lesson from this patient encounter – not to be ruled by my prejudices (student 12)

Some students had identified ethical problems of clinical practice, mainly concerning patients’ autonomy. Comments on future communication skills training were centred on how to practise getting patients involved in decision making and how to further develop motivational interviewing.

Content

A third of students had been required to recapitulate their knowledge of basic and clinical science in a vast range of medical conditions, consolidating previous knowledge and adding new facts related to their patient presentations. Only a few students had conducted a more thorough search of the literature in connection with problems they had come across in their patient encounters. Several students reflected on their plans to acquire further knowledge, striving constantly to keep their knowledge base up to date.

Students’ evaluation questionnaire

The 35 students all supplied their evaluations and presented numerous free comments. They were all satisfied with their mentors’ support, and all students except one were comfortable with the feedback provided by the examiner in the final individual interview. Sixteen students found the instructions vague or ambiguous. The most positive experience recognized was the final interview with the examiner: ‘the feedback was fantastic and quite necessary for compiling a meaningful portfolio’. Almost all students also emphasized the opportunity for reflective writing as positive: ‘good to be ‘forced’ to reflect in a structured way on your strengths and weaknesses’, ‘learned several strategies to improve’, ‘great to see my development during the course’. The negative experiences were centred on the clearly deficient instructions in the portfolio, and some students also found the portfolio time-consuming. Despite this criticism, students strongly recommended us to launch the portfolio, with improved instructions, for all students in the programme.

Discussion

This study explored the themes of portfolio reflections by a group of final-year medical students, undertaking a GP attachment. Notwithstanding the standardized format and the limited time available, students provided good evidence of their learning. We were encouraged that a wide variety of dimensions of professional competence was provided by students’ reflections. The portfolio was popular with participating students.

The strength of our study is the insight into the content and nature of students’ reflections, an issue only sparsely discussed in previous research (Buckley et al. 2009). Our study adds to the existing knowledge of students’ learning experiences in GP in three principal fields: consultation skills, patient-centred care, and professional issues.

Firstly, students’ reflections on their consultation skills showed that the most important learning issue in the theme of ‘process skills’ was how to achieve structure in the medical interview. Patient-centred communication can cause genuine concerns in students about losing control of the consultation and getting caught in a flow of less ordered information from the patient (Silverman et al. 2004). When students learned how to obtain structure, they also experienced better time management, enhanced self-confidence and an improved rapport with the patient.

Secondly, the theme of ‘patient-centred care’ was emphasized in students’ case summaries. Patient-centred care is a major principle in GP, but is of essential interest to several other medical disciplines as well. Achieving shared decision making, understanding the patient as a whole person and exploring the patient's perspective are aspects shown to positively influence patients’ satisfaction and enablement, reduce their symptom burden and discomfort, and increase efficiency of care (Stewart et al. 2000; Little et al. 2001). We believe that these central dimensions of patient-centred care ought to be highlighted earlier in hospital clerkships.

Thirdly, the findings suggested integration of the cognitive, affective and practical dimensions of professional competence that can be acquired in a clinical learning environment (Mann et al. 2007; van Tartwijk & Driessen 2009; Passi et al. 2010). As expected, students reflected on their ‘content skills’ and provided evidence for their identification of deficits in knowledge and skills, their integration of new understanding, and their plans for development. However, ‘perception skills’ was a more prominent theme in both units of analysis. In their self-awareness of feelings, attitudes and concerns, students were unexpectedly open and frank about their deficiencies and about how they had tried to remedy them. They also acknowledged the importance of ethical issues and additional communication skills training, and openly reflected on their clinical reasoning. However, we found it remarkable that several students, in their final year in medical school, reflected on their initial problems in transferring theory to practice; a skill supposed to be frequently trained during previous hospital clerkships.

The main limitations of our study are the single group design and the relatively small group of participating students, who also were self-selected volunteers. The pilot naturally attracted students who were more prone to self-reflection and more capable in writing. However, it is likely that other students participated out of curiosity, excitement about a new intervention or attracted by the prospect of a shorter written test, as the results of students’ efforts varied considerably. Another limitation is the fact that the study was conducted in one single institution, and the results of this qualitative study cannot be generalized to all other institutions or medical schools.

The final examiner interview and the support by academic teachers as mentors strongly contributed to students’ positive evaluation of the portfolio. The interview also gave the examiners an opportunity to enable students to take their reflections to a deeper level and to supply feedback (Friedman Ben David et al. 2001). We believe that these factors also were crucial to students’ openness in their reflections. Grant et al. (2007) described a GP portfolio with GP supervisors as mentors and without a final interview, and reported on students’ perceived limited benefits from learning and on their unwillingness to discuss their feelings in their reflections.

The utility of the portfolio assessment can be discussed by using a conceptual model with five variables: validity, reliability, educational impact, acceptability and cost (van der Vleuten 1996). Face validity in portfolio assessment can hardly been contested. We also believe that the content validity (credibility) of the assessment structure is ensured by clear learning outcomes, a variety of written evidence of students’ experiences and reflections, extended placement in practice, daily observation by practice supervisors and documents signed by supervisors (Webb et al. 2003). The qualitative analysis of themes in students’ reflections, linked to the documentary evidence, can also ensure content validity.

The reliability (dependability) of the assessment process is strengthened by the verification of documents by the examiner, explicit assessment criteria (pass/fail) and an ‘external’ examiner (Webb et al. 2003). Standardization of portfolio content, discussion between examiners, and the personal examiner interview also contributes to reliability. However, the inter-reliability of the assessment warrants further psychometric evaluation.

As to the educational impact, we can hardly maintain more than the lowest level (‘reaction’) of Kirkpatrick's hierarchy, measuring students’ satisfaction in their evaluation (Kirkpatrick 1996). The analysis of the themes in students’ reflections assured us of improvements in their knowledge and understanding and their enhanced self-awareness, but these issues were not objectively measured.

The four teachers who took part in the pilot, in their combined roles of mentors and examiners, found the portfolio pilot an enjoyable experience. The reviewing of the portfolios and the individual interviews were regarded as a worthwhile use of their time, and the costs of the assessment were acceptable.

Conclusions

In summary, this study demonstrated that a structured portfolio in a short timeframe provided abundant opportunities for students to reflect on personal and professional issues, notwithstanding the standardized format. We strongly recommend a final examiner interview to deepen students’ reflections, to supply feedback and to raise the importance of reflective ability as a skill in future learning from practice.

Acknowledgement

We wish to remember Anders Håkansson, professor of GP, recently deceased, who significantly contributed to the design and accomplishment of this portfolio pilot.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.