Combined formative and summative professional behaviour assessment approach in the bachelor phase of medical school: A Dutch perspective

Figures & data

Table

Practice points

One should realise that approaches to the concepts of professionalism and PB as well as legal possibilities for dismissal of dysfunctioning students differ not only between countries, but also within continents. Likewise, the approach to assessment may differ. Although the focus tends to be on the summative aspects of PB assessment, it seems perfectly feasible to combine teaching, as well as formative and summative assessment in the same procedural approach. PB assessment may be performed combining judgements during regular educational activities with critical incidents regarding unprofessional behaviour. Although many students occasionally receive a negative judgement during regular educational activities, the number of persistently dysfunctioning students after counselling and guidance is limited. So far, dismissal of persistently dysfunctioning students is not possible within Dutch legal regulations.

Table 1.  Dimensions of PB as defined by the Project Team Consilium Abeundi of the Association of Universities in the Netherlands (Project Team Consilium Abeundi van Luijk 2005) (reproduced with permission from van Mook et al. 2009e)

Dimension	Aspects	Subcategory	Details
Dealing with tasks/work	Time management Information management Ability to work independently Requesting supervision when necessary Preparation and presentation of patient demonstration Punctuality Honouring commitments Observable dedication, commitment and sense of responsibility Coping with stress Leadership	Monitoring level of competence	Keeping up to date in the professional field, loyally to the profession, responsibility and independence
		Monitoring limitations of competence	Indicating where competence ends, abstaining from behaviour/statements that are not within one's competence, even when colleagues put pressure on one to exceed these limitations
		Participation in team competence	Forms of teamwork to achieve team competence, delegating and division of tasks, chairing meetings, substituting for others, honouring commitments, loyalty to decisions made
Dealing with others	Communicating in understandable language Adequate command of Dutch language Adequate non-verbal behaviour Adequate behaviour in gathering and giving information Giving a presentation/reporting Ability to construct a structured consultation Honouring commitments Collaborating Dealing with privacy and doctor patient confidentiality Empathising with patient's expectations Clearly explaining one's role as clerk and its possibilities and limitations Handling conflict Handling emotions of others Adequate interaction with colleagues and nurses Being courteous and respectful Negotiating skills	Gathering information from another person's perspective	Includes empathy, being unprejudiced, finding contextual information, active awareness of the emotional implications of information
		Providing information from another person's perspective	All behaviours directed at information and advice tailored to the recipient: e.g. adapting information to the other person's emotions and capabilities for understanding, presenting concrete information, checking whether the information has actually been received, inviting response
		Aligning other person's perspective and one's own competence in decision making	All behaviours directed at facilitating decision making, taking into account both the other person's perspective and the doctor's competence. E.g. encouraging someone to respond, meta-communication, implementation of informed consent procedures
Dealing with oneself	Ability to self-reflect Dealing with feedback and criticism Good personal appearance Self-management/setting boundaries Dealing with uncertainty Handling own emotions	Reflection on own behaviour and the underlying dynamics	Self-observation, gaining insight into and dealing with one's emotions, motivation, cognitions including values, standards, prejudices, as well as how these have evolved, and their effect on one's own behaviour
		Testing one's reflections against the opinions of others	Being receptive to and stimulating feedback on the and criticism on one's own behaviour
		Implementation of the results of reflections against the opinions of others	Developing and experimenting with behavioural alternatives on the basis of self-reflection and feedback

Figure 1. The PB assessment form currently in use in Faculty of Health, Medicine and Life Sciences, Maastricht.

Figure 2. Algorithm displaying the procedure for dealing with the judgements of students’ PB after all scheduled educational activities, as well as critical incident reports.

Figure 3. Algorithm displaying the final (end-of-year) judgement procedure regarding.

Table 2.  Lessons learned regarding teaching and assessment of PB

No.	Tip
1	Create awareness regarding the importance of PB in students early in the curriculum, for example by giving a lecture or implementing a white coat ceremony.
2	Aspects of medical history, ethics, sociology, humanism, law and other areas where professionalism issues are represented, should be dispersed throughout the curriculum.
3	The formative aspect of PB assessment is obviously essential, and should compliment summative PB assessment.
4	Use PB forms which provide ample space for narrative evaluative feedback, which faculty are highly encouraged to complete.
5	Create awareness regarding the importance of PB assessment by instituting mandatory staff development programs, and making it a mandatory and scheduled part of each course.
6	In case of doubt between ‘pass’ or ‘fail’ the committee urges staff to preferably give the judgement ‘fail’ over ‘pass-by-mercy’.
7	The tutor fulfils a pivotal role regarding PB assessment, and largely determines the quality of PB assessment. The instrument itself (the form) itself has limited intrinsic value.
8	Development of additional training programmes (such as the described CORE program), underscoring the importance of aspects of professionalism and emphasis on PB development can compliment professional development during regular educational activities.
9	Since every professional has a range of behaviours, incidental occurrences of inappropriate behaviour (so-called lapses) are not per se synonymous with unprofessional behaviour.
10	Early lapse identification is critical to facilitate remediation attempts before behaviour becomes refractory to change
11	Furthermore, a negative judgement regarding PB should never be compensated for by other aspects of clinical competence.
12	A clear remediation plan should be individually tailored to the student's needs, and may involve alternating meetings with for example the student advisors, the CPB and a psychologist, with regular information exchange between all those involved.
13	Practical aspects are having clear objectives before scheduled meetings, having witnesses present, scribing, and setting transparent goals and expectations.
14	Any progress reported and agreements made, should be tested against the original plan of action.
15	A timeframe in which these should be fulfilled, and assuring adequate guidance and frequent performance feedback sessions are also of paramount importance.
16	Nevertheless, all reports should be confidentially handled, and complainants should receive follow-up of review and resulting actions.
17	An algorithmic advice (summation and subtraction of number of negative and positive judgements) should be abandoned for several reasons.
18	Changes will certainly not occur in the absence of strong institutional leadership!

Table 3.  Overview of numerical data on negative PB judgements during regular educational activities and critical incident reports in the period 2005–2009 for the first 2 years of Faculty of Health, Medicine and Life Sciences, Maastricht, The Netherlands

Year 1
	Number of students in year 1				Number of judgements				Number of students involved				Gender of students in year 1 Male/Female
Judgements during scheduled educational activities	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009
Fail					10	23	23	30	10	19	20	26
No judgement possible					78	73	82	147	54	56	58	88
(Sub)total					88	96	105	177	64	75	78	114
Incidental reports regarding critical incidents					3	6	6	21	2	6	6	18
Total	359	355	341	375	91	102	111	198	66	81	84	132	131/228	115/240	119/222	147/228
	Gender of students involved
	Male				Female				Meeting with Committee on PB				Referral student counsellors
Judgements during scheduled educational activities	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009
Fail	8	7	11	21	2	12	9	5	9	16	12	19	3	5	4	9
No judgement possible	28	23	3	49	26	33	1	39	5	1	1	7	1	0	0	0
(Sub)total	36	30	14	70	28	45	10	44	14	17	13	26	4	5	4	9
Incidental reports regarding critical incidents	2	4	4	14	0	2	2	4	1	3	1	3	1	3	0	2
Total	38	34	18	84	28	47	12	48	15	20	14	29	5	8	4	11
Year 2
	Number of students in year 2				Number of judgements				Number of students involved				Gender of students in year 2 (Male/Female)
Judgements during scheduled educational activities	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009
Fail					28	21	25	21	24	21	24	18
No judgement possible					75	104	92	97	56	72	71	77
(Sub)total					103	125	117	118	80	93	95	95
Incidental reports regarding critical incidents					3	5	2	12	3	5	2	12
Total	367	359	344	326	106	130	119	130	83	98	97	107	149/218	141/218	121/223	117/209
	Gender of students involved
	Male				Female				Meeting with Committee on PB				Referral student counsellors
Judgements during scheduled educational activities	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009	2005–2006	2006–2007	2007–2008	2008–2009
Fail	18	10	16	14	6	2	8	4	13	6	13	11	3	2	3	3
No judgement possible	30	41	40	42	26	31	31	35	15	6	9	4	2	3	0	2
(Sub)total	48	51	56	56	32	33	39	39	28	12	22	15	5	5	3	5
Incidental reports regarding critical incidents	2	1	1	10	1	4	0	2	2	1	0	030	2	2	0	1
Total	50	52	57	66	33	37	39	41	30	13	22	15	7	7	3	6

Table 4.  Illustrative examples of negative PB judgement during a regular educational course, an isolated critical incident report, as well as multiple and persisting negative PB judgements despite remediation attempts

Box	Example
1	Isolated negative PB judgement during a regular educational course
	Student A was given a negative PB judgement by the tutor during a 6-week course in the second year for reasons of unexplained absence during a tutorial group meeting. The student was previously known with neither PB, nor knowledge and skills deficits. During the CPB meeting the student explained that absence during 2 tutorial group meetings was unavoidable due to participation in the management of an international student association. This absence had been cleared with the tutor prior to the meeting. Returning from the meeting student was confronted with the severe illness of a relative, which he subsequently visited. Student failed to inform the tutor of this absence, considering it merely a private matter. Soon thereafter the relative passed away. The funeral was not attended since it would have resulted in additional absence. In retrospect the student regrets non-informing the tutor, designates it as unprofessional and not acceptable. Honesty would have been more appropriate. The CPB empathises with the reasons for the absence, but prior explaining would have contributed to the tutor's understanding. The advice was given to communicate more clearly with staff regarding such matters. No further negative PB judgements have been received thereafter.
2	Isolated critical incident report
	The CPB received a critical incident report on student B written by the mentor and CORE-coach. For reasons of absence during simulated patient contacts and their subsequent discussion during the CORE meetings, and related insufficient communication with the coach and peers. After several remediation attempts the student was refused further participation in the CORE programme. The student responded in writing, and later before the CPB, that the student struggled with chronic health problems. Complaints, medical testing, treatment and doctors’ visits limited the ability to participate in the training programmes. The student was not aware of special arrangements for students with a chronic illness. Referral to the student advisors and dean for student affairs resulted in less strict attendance requirements, pre-registration possibilities for skills training and tutorial groups, and individual examination possibilities of extended duration. The CPB advised to re-discuss the persisting health problems with the mentor and CORE-coordinator, and be frank to peers regarding possible future limitations to peers. Ultimately re-entering of the CORE programme was not feasible, and the student repeated the programme a year later.
3	Multiple persisting negative PB judgements despite remediation attempts
	Student C was repeatedly given negative PB judgements as well as received multiple critical incident reports during the first, second and third years. This pattern was paralleled by frequent and repeated failure on knowledge and skills testing. Passing the first two year of medical school took more than a lustrum. Major problems identified were communications problems and not being able to adequately cooperate regarding assignments with peers, lack of self-reflection, repeatedly ignoring feedback, perception of intentional humiliation and insulting by tutors, aggressive writing style when responding to questions by tutors and peers, criticising tutors and coaches when provided with constructive feedback, not showing up for resits and too frequent unauthorised absence. Despite intensive guidance by the student advisor, no improvement was observed. A comparable pattern was observed during correspondence and meetings with the EC and CPB. Reponses were primary, with an unpleasant, pejorative undertone which seemed to confirm instead of refuting the assessors’ views. Although communication during meetings was less hostile, e-mails sent to the EC and CPB, often without delay, accused them of e.g. ‘wilful bullying, intimidation, abuse of power and rights, and willingly obstructing study progress’. Repeatedly, an advice to end the study was not appreciated. Since mandatory cessation of study is so far impossible in the Netherlands, expiring study results ultimately formed the basis for preventing graduation from medical school.

Table 5.  Overview of end-of-year judgements in the period 2005–2009 for the first two years of Faculty of Health, Medicine and Life Sciences, Maastricht, The Netherlands

Year 1
	Number of judgement (%)
Final end-of-year judgement	2005–2006	2006–2007	2007–2008	2008–2009
Pass	326 (96)	319 (93)	302 (92)	297 (90)
No judgement possible	9 (3)	20 (6)	21 (6)	16 (5)
Fail	3 (1)	3 (1)	5 (2)	17 (5)
Total	338 (100)	342 (100)	328 (100)	330 (100)
Year 2
Pass	331 (94)	314 (93)	310 (93)	278 (94)
No judgement possible	14 (4)	19 (6)	19 (6)	16 (5)
Fail	6 (2)	5 (1)	2 (1)	4 (1)
Total	351 (100)	338 (100)	321 (100)	298 (100)

van Mook W, van Luijk S, O'sullivan H, Wass V, Zwaveling J, Schuwirth L, van der Vleuten C. The concepts of professionalism and professional behaviour: Conflicts in both definition and learning outcomes. Eur J Int Med 2009e; 20: e85–e89

 PubMedGoogle Scholar