Introduction
A reform of specialist training for general practice is taking place in Denmark Citation[1]. One consequence of the reform is a mandatory personal portfolio, which contains registrations of the trainee's clinical performance and experience in relation to the qualifications needed for a specialist in general practice.
A significant body of literature describes the use of portfolios in medical education Citation[2–9], but so far the practical setup has been given little attention. This article describes an online portfolio for specialist training in family medicine in Denmark Citation[10]. The focus of this article is on the practical setup and step-wise development guided by evaluations and feedback from users. Effects of the portfolio on learning have been reported elsewhere Citation[11].
Development of the portfolio
We used an “action research like approach” to develop the portfolio, divided into five phases: a definition phase, an exploratory phase, an improvement phase, an evaluation phase, and a production phase.
Definition phase
Several working groups within the Danish College of General Practitioners developed a blueprint defining the competences needed for a specialist in general practice Citation[11]. The process was based on registered daily medical activities in general practice and supported by consultants from other specialities. A prototype of the online portfolio was based on the blueprint and launched in September 2003. GP trainees and GP trainers were instructed by mail to use the online portfolio but received no formal introduction about its use.
Exploratory phase
Experience was collected by means of group discussions amongst the first 10 pairs of trainers and trainees (focus groups) Citation[12] and by written evaluations from the next 6 GP trainees. These exploratory data were analysed by the authors and validated by sending the conclusions to all the participating doctors and asking for their comments.
Data from group discussions and open-ended written evaluations could be categorized into: “roads to learning”, “usability”, “access to data”, and “obstacles”.
They revealed four feasible roads to reflective learning; a portfolio should support:
reflection on significant clinical events;
committed learning (individual learning contracts);
personal learning diary (a personal journal for reflections, extracurricular activities among other things);
horizontal learning – peer-to-peer learning (experience exchange by trainees in different GP surgeries).
Representative quotation from a trainee: The patients set the agenda … . I have to learn from “the patient of the day” not from a goal in a book; a portfolio has to respect this.
Improvement phase
Based on these data, a new format for the portfolio was constructed and launched.
Evaluation phase
In order to evaluate the new format, a questionnaire was constructed to collect quantitative and qualitative data. It was validated for construct and content validity by an external process and tested for ambiguous questions by 6 trainees in a pilot study.
From September 2003 till March 2005, 315 trainees registered for the portfolio. The questionnaire was sent to the first 90 registered trainees. Only 79 of them were users, defined as trainees who had visited the portfolio online more than four times within the first two months. The response rate among the users was 71% (56/79) after one reminder.
The three researchers divided the qualitative responses to the questions into “meaning-carrying units” by a phenomenological method Citation[13] in a process allowing researcher triangulation.
Data were condensed into themes as listed in Box 1.
Box 1. User recommended facilities for an online portfolio
Web access with broadband internet connection from the working station of the trainee in practice and at home
Highlighting of active objectives
Efficient and “intelligent” search function which allows search with synonyms
Space for written reflections linked to one or several specific learning objectives
A print function
Space for personal notes and link collections
Ability to highlight newly written notes for the trainer
Easy distinction between notes made by the trainer and the trainee
Support of trainer assessment
Notes from the trainer should be linked to specific learning objectives or notes made by the trainee
Possibility of moving clinical notes from Electronic Patient Record to the portfolio by copy/paste
Possibility of synchronisation with a PDA and ability to use in an offline version (e.g. for pendlers)
Possibility of exchange of experiences among trainees placed on different locations and educational levels
Online introduction
Technicalities and programme recommendations
In order to make a portfolio operational it should support the four roads to reflective learning as mentioned earlier and it should contain a number of facilities:
a part for private reflections by the trainee, as a personal journal or learning diary;
a part shared by the trainee and the trainer, aimed at formative feedback and mutual reflection;
a part with aims and objectives shared with the health authorities (blueprint and approved learning objectives);
a part showing the educational progress, to assist educational management.
an efficient electronic search function that allows the trainee to identify the learning objectives related to specific significant events;
a platform, which allows the exchange of electronic information, such as email, sms, PDA, FAX etc.;
horizontal learning – peer-to-peer learning (experience exchange by trainees in different GP surgeries).
A list of further specific programme recommendations is listed in Box 2.
Box 2. Important points when planning to introduce an online portfolio
Choose a format supportive of reflective learning and not simply data collection
Choose a format constructed in accordance with learning by experience, reflection, and feedback
Choose an online format
Ensure that time and resources are available for proper introduction of both trainers and trainees
Ensure protected time is at available during daily working hours; the use should be additional to normal trainee–trainer interaction – not a substitute
Ensure active participation of both trainees and trainers
Allow a private part exclusive to formative feedback if other parts of the portfolio are to be used for summative assessment
Representative quotation from a trainee: In my first job we only had a poor internet connection, but in my present job we have a broadband connection, which facilitates the use a lot; I am sure it [the portfolio] will turn out be a proper tool.
User acceptance and obstacles
The usability was regarded as high. The main problems were too little time, missing trainer support, lack of proper introduction, and missing personal motivation.
Almost all (96%) of the users expected to use the portfolio in the future. They clearly preferred an online version to a pen-and-paper version ().
Representative quotation from a trainee: A portfolio needs time, protected time in a busy day, I think approx. 10–15 minutes daily, these can be difficult to find, and time … . You have to start using the portfolio on day one if the outcome is to be optimal.
Table I. Selected questions from the user questionnaire.
Production phase
Based on the data from the evaluation phase a new revision of the portfolio was made. The portfolio is accessed via the internet. The learning objectives are reached via entrances, supporting different types of reflective learning. The trainee divides the data reported to the portfolio into a part for private reflections and a part shared with the trainer, aimed at formative feedback. The portfolio has an efficient search function, which allows the trainee to identify the learning objectives related to specific significant events. The portfolio shows the approved objectives and the educational progress, to assist educational management () (a demo version can be seen on http://www.logbog.net/uk).
Figure 1. The front page of the Danish portfolio with an overview of the content.
Discussion
We found that interaction with users is important when designing an acceptable and suitable online portfolio. However, programme design and usability are not the only factors of importance for the use of a portfolio. Lack of protected time (time set apart for the purpose) will be an important barrier preventing the use of any portfolio. Active involvement of trainee as well as trainer and a proper introduction appear to be as important as programme design.
This article focuses on the development of the portfolio and is based on evaluations from users. Non-users are an equally interesting group, who experienced obstacles such as lack of time, lack of trainer support, inappropriate IT facilities, lack of proper introduction, and lack of personal motivation Citation[11].
Our results on a suitable design and conditions for successful use can be understood in a constructivist learning theoretical perspective Citation[14–17] and are in accordance with the literature Citation[2], Citation[3], Citation[9], Citation[18], Citation[19].
References
- Kjaer NK, Tulinius C. Learning in General Practice in Denmark. Two master theses, the MHPE programme, 2003. Maastricht University. Available at: www.logbog.net.ref.
- Challis M, Mathers NJ, Howe AC, Field NJ. Portfolio-based learning: Continuing medical education for general practitioners – a mid-point evaluation. Med Educ 1997; 31: 22–6
- Mathers NJ, Challis MC, Howe AC, Field NJ. Portfolios in continuing medical education: Effective and efficient?. Med Educ 1999; 33: 521–30
- Snadden D, Thomas ML, Griffin EM, Hudson H. Portfolio-based learning and general practice vocational training. Med Educ 1996; 30: 148–52
- Snadden D, Thomas ML. Portfolio learning in general practice vocational training: Does it work?. Med Educ 1998; 32: 401–6
- Wilkinson TJ, Challis M, Hobma SO, Newble DI, Parboosingh JT, Sibbald RG, Wakeford R. The use of portfolios for assessment of the competence and performance of doctors in practice. Med Educ 2002; 36: 918–24
- Pearson DJ, Heywood P. Portfolio use in general practice vocational training: A survey of GP registrars. Med Educ 2004; 38: 87–95
- Webb C, Endacott R, Gray M, Jasper M, Miller C, McMullan M, Scholes J. Models of portfolios. Med Educ 2002; 36: 897–8
- Dagley V, Berrington B. Learning from an evaluation of an electronic portfolio to support general practitioners’ personal development planning, appraisal and revalidation. Education for Primary Care 2005; 16: 567–74
- Eklund-Grönberg A, Stange P. News from the Nordic Colleges of General Practitioners: The future is approaching. Scand J Prim Health Care 2006; 24: 193–5
- Kjaer NK, Maagaard R, Wied S. Using an online portfolio in postgraduate training. Med Teach 2006; 28: 708–12
- Coar L, Sim J. Interviewing one's peers: Methodological issues in a study of health professionals. Scand J Prim Health Care 2006; 24: 251–6
- Giorgi A. Sketch of a psychological phenomenological method. Phenomenology and psychological research, A Giorgi. Duquesne University Press, Pittsburgh 1985; 126–8
- Schön DA. The reflective practitioner: How professionals think in action. Aldershot: Ashgate/Arena, 2000/1983.
- Schmidt HG, Norman GR, Boshuizen HPA. A cognitive perspective on medical expertise: Theory and implications. Acad Med 1990; 65: 611–21
- Fish D, Coles C. Developing professional judgment in health care: Learning through the critical appreciation of practice. Butterworth Heinemann, Oxford 1998
- Kjaer NK. Between science and practice, a theoretical perspective on learning] (in Danish). Ugeskr Laeger 2003; 165: 397–400
- Dornan T, Carroll C, Parboosingh J. An electronic learning portfolio for reflective continuing professional development. Med Educ 2002; 36: 767–9
- Svenberg K, Wahlqvist M, Mattsson B. “A memorable consultation”: Writing reflective accounts articulates students’ learning in general practice. Scand J Prim Health Care 2007; 25: 75–9