The development of an instrument for evaluating clinical teachers: involving stakeholders to determine content validity

Figures & data

Table

Practice points
•The cognitive apprenticeship model appears to offer a useful framework for designing an evaluation instrument aimed at providing feedback to clinical teachers on their supervision of medical students.
•The clinical supervisor evaluation instrument is considered relevant by three groups of stakeholders in undergraduate clinical training (educationalists, doctors and students).
•The positive rating of the instrument's content validity by three stakeholder groups suggests that the instrument will meet with broad acceptance when it is implemented in clinical training.
•Future research should focus on further validation of the instrument, on effective ways of providing feedback to clinical teachers based on student evaluations and on promoting faculty development based on student evaluations.

Table 1.  Mean ratings and SDs, on a Likert scale (1 = highly irrelevant; 5 = highly relevant), of the relevance of the items relating to eight supervision methods by stakeholder group and for the three stakeholder groups combined and overall group ratings per supervision method

	Educationalists		Doctors		Students		Total item score
Supervision methods	M	SD	M	SD	M	SD	M
Modelling
The clinical teacher:
1. demonstrated how different tasks should be performed.	4.5	0.53	4.5	0.63	3.6	0.80	4.2
2. explained, while performing a task, which aspects were important and why.	4.4	0.84	4.7	0.48	4.2	1.00	4.4
3. created sufficient opportunities for me to observe him or her.	4.4	0.70	4.4	0.51	3.8	0.87	4.2
4. was a role model for me.	4.3	1.10	4.3	0.68	3.2	1.00	3.9
Overall ratings on modelling	4.4	0.41	4.5	0.45	3.7	0.47
Coaching
The clinical teacher:
5.  observed me while I was performing a task.	4.9	0.32	4.5	0.63	4.2	0.83	4.5
6.  provided me with constructive and concrete feedback during or following direct observation.	4.6	0.70	4.7	0.48	4.5	0.70	4.6
7.* provided me with constructive and concrete feedback during my rotation.	4.3	1.10	4.5	0.90	4.7	0.50	4.5
8.  gave me a better insight into aspects of my performance that needed improvement.	4.3	0.80	4.6	0.63	4.0	0.85	4.3
Overall ratings on coaching	4.5	0.50	4.6	0.60	4.3	0.50
Scaffolding
The clinical teacher:
9. adjusted his/her teaching activities to my level of experience and competence.	4.1	1.00	4.2	0.75	3.8	0.62	4.0
10. allowed me to perform tasks that fit my level of experience and competence.	4.3	1.10	4.4	0.63	4.2	1.10	4.3
11. was supportive when I experienced difficulties with a task.	4.0	1.20	4.5	0.73	4.1	0.51	4.2
12. gradually decreased the amount of guidance in order to bolster my independence.	3.4	1.10	4.5	0.52	4.3	0.75	4.1
Overall ratings on scaffolding	4.0	0.66	4.4	0.47	4.1	0.44
Articulation
The clinical teacher:
13. asked me to explain my reasoning and arguments.	4.6	0.70	4.6	0.63	4.5	0.52	4.6
14. alerted me to gaps in my knowledge and skills.	4.2	0.63	4.4	0.62	3.9	1.10	4.2
15. asked questions to increase my understanding.	4.3	0.71	4.6	0.62	4.5	0.70	4.5
16. stimulated me to ask questions to increase my understanding.	4.5	0.71	4.6	0.50	4.0	1.10	4.4
Overall ratings on articulation	4.0	0.50	4.5	0.48	4.4	0.57
Reflection
The clinical teacher:
17.  stimulated me to think about my own strengths and weaknesses.	4.4	0.84	4.4	0.62	3.4	0.90	4.1
18.  stimulated me to think about how to improve my own strengths and weaknesses.	4.5	0.71	4.6	0.63	3.4	1.10	4.2
19.* stimulated me to compare my own approach to that of a more experienced doctor.	3.9	1.20	3.7	1.10	2.8	0.87	3.4
20.* stimulated me to reflect on the profession of a medical doctor.	3.5	0.85	3.8	1.00	3.0	1.30	3.4
Overall ratings on reflection	4.4	0.66	4.1	0.70	3.1	0.63
Exploration
The clinical teacher:
21.  stimulated me to formulate my own goals.	4.7	0.50	4.1	0.85	3.5	0.67	4.1
22.  stimulated me to achieve my own goals.	4.8	0.42	4.3	0.45	3.8	0.75	4.2
23.  challenged me to explore new tasks and possibilities.	4.5	0.85	3.9	0.72	4.3	0.75	4.2
24.* challenged me to extend my boundaries.	3.6	1.50	3.8	0.83	4.1	0.90	3.8
Overall ratings on exploration	4.5	0.41	4.0	0.59	3.9	0.54
General Learning Climate
The clinical teacher:
25. established a safe-learning environment.	4.4	0.84	4.8	0.45	4.1	0.67	4.4
26. showed an interest in me as a student.	4.4	0.70	4.7	0.48	4.2	0.83	4.4
27. treated me with respect.	4.7	0.50	4.9	0.34	4.3	0.62	4.6
** took enough time to supervise me.
Overall ratings on general learning climate	4.3	0.50	4.8	0.38	4.2	0.56
Miscellaneous
28. Give an overall mark (1–10) for this doctor as a clinical teacher.	3.7	1.60	4.2	0.83	3.6	1.20	3.9
29. What are the strengths of this clinical teacher? (open-ended question)	4.7	0.49	4.6	0.62	4.4	0.52	4.6
30. Which aspects of the performance of this clinical teacher can be improved? (open-ended question)	4.7	0.52	4.8	0.40	4.3	0.67	4.6
Overall ratings on miscellaneous	4.3	0.85	4.5	0.36	4.1	0.49

*= Eliminated item; ** = added item.

Table 2.  Pearson chi-square and effect sizes (ratings of relevance per teaching method for three groups of stakeholders)

Teaching method	χ^2	df	Effect size
Modelling	33.521*	6	0.47
Coaching	8.276	4	–
Scaffolding	23.004*	6	0.39
Articulation	5.162	4	–
Reflection	27.123*	6	0.42
Exploration	28.255*	6	0.43
General learning climate^a	–	–	–
Miscellaneous	6.806	4	–

^aPreconditions for Pearson chi-square not met.

*Bonferonni corrected significance level p < 0.01.