Student selected components: do students learn what teachers think they teach?

Abstract

Background: It is well recognized that what teachers teach and what students learn may not be the same. This applies to all parts of the undergraduate medical curriculum, but may be especially relevant to student selected components, which vary substantially in their educational content. This has not been studied previously.

Aims: To compare perceptions of students and supervisors in relation to learning outcomes addressed by student selected components, and thus to examine differences between what is taught and what is learned.

Methods: Supervisors (n = 69) were asked to indicate which of twelve learning outcomes they felt were components of teaching and assessment. Upon completion of each SSC, students were required to complete the same outcomes template as part of their feedback (n = 644). Perceptions were compared in two ways: (1) a colour-coded ‘traffic-light’ system was used to record agreement/disagreement between students and supervisors of individual SSCs; (2) differences in perception of outcomes across the entire SSC programme were compared using the χ² statistic.

Results: (1) The ‘traffic-light’ system readily identified individual SSCs where significant disagreement existed and which were subject to further scrutiny. (2) More students than supervisors thought that outcome 2 (competent to perform practical procedures) was a component of teaching and assessment (41.8% v 27.5%, χ² = 5.24, p = 0.02), whereas more supervisors than students thought that outcome 6 (competent in communication skills) (97.1% v 82.1%, χ² = 6.91, p = 0.009) and outcome 7 (competent to retrieve and handle information) (100% v 93.7%, χ² = 4.8, p = 0.02) were.

Conclusions: Significant disagreement exists about the outcomes addressed by SSCs, suggesting that students do not always learn what teachers think they teach. The use of two complementary approaches allows global and individual comparisons to be drawn and thus provides a powerful tool to address this important issue.

Introduction

Most undergraduate medical courses in the UK now include student selected components (SSCs). Briefly, these are courses or modules selected by students from a menu of options provided by staff separately from the ‘core’ curriculum. The main driver for the inclusion of SSCs into undergraduate medical teaching was the publication in 1993 of Tomorrow 's Doctors by the General Medical Council (GMC 1993); this document was subsequently updated (GMC 2002). There is considerable heterogeneity both within and between SSC programmes, and this underpins the need for the educational content of individual SSCs to be identified. One way of doing this is to specify learning outcomes that are components of teaching and assessment. Such an outcomes-based approach to SSCs mirrors similar developments in relation to the ‘core’ curriculum (Harden et al. 1999). We have used such an approach at the University of Dundee Medical School for the past five years. A learning outcomes template (the ‘Dundee learning outcomes’) is applied to all parts of the curriculum including SSCs. We require all staff offering SSCs to specify which outcomes are components of teaching and assessment. We have taken the opportunity presented by the requirement for student feedback to compare the perceptions of students and supervisors in relation to outcomes addressed by SSCs.

Table

Practice points

This is the first publication to provide data about student learning in SSCs. The study provides evidence for a discrepancy between the ‘planned’ and ‘learnt’ curriculum. Two approaches are described for comparing student perceptions of what they have learned with teacher- intended learning outcomes: (a) a ‘traffic light’ system for analysis of learning in individual SSCs; and (b) a global system for analysis of learning outcomes across an entire SSC programme.

Background

SSCs completed in 2005/06 by medical students in Phase 2 of the Dundee curriculum form the basis of the current study. Phase 2 then consisted of the second and third years of a five-year curriculum. Across the two years, sixteen weeks were devoted to SSCs. All SSCs were either two or four weeks long. Thus individual students completed between four and eight SSCs depending on the length of their preferred SSC choices. Students were allowed to choose from a repertoire of organized SSCs, or to self-propose their own SSC. The current report relates specifically to organized SSCs; self-proposed SSCs were not included in the analysis. The range of SSCs completed was extremely varied. Some covered core topics in more depth, e.g. clinical and basic sciences; others covered medical topics related to the core, e.g. sports medicine, history of medicine, health policies; yet others topics less directly related to medicine e.g. medical French or Spanish.

Methods

Supervisors were asked when offering their SSC to indicate which of the twelve Dundee learning outcomes they felt were addressed by their SSC and, if so, whether they thought these were major or minor components of teaching and assessment. The relevant section of the documentation completed by supervisors is shown in Figure 1. Upon completion of each SSC, all students were required to complete the same outcomes template as part of their feedback.

Figure 1. Outcomes section of SSC intent form.

Comparison of perceptions

Two approaches were used to compare perceptions of supervisors and students.

Individual SSCs

Perceptions of individual SSCs were compared by means of a template, using a system of colour-coding analogous to traffic-lights. If supervisor and student agreed completely about the place of a particular outcome in teaching and assessment, the relevant cell was colour-coded green. If they disagreed completely, the cell was colour-coded red. This occurred if one thought an outcome was a major component of teaching and assessment, while the other did not think it was addressed at all. If there was disagreement about whether an outcome was a major or minor component of teaching and assessment, the cell was colour-coded amber (partial disagreement). A second scenario was also colour-coded amber, namely where one thought an outcome was a minor component of teaching and assessment, while the other did not think it was addressed at all.

Figure 2 illustrates this basic template. Learning outcomes are numbered on the horizontal axis, and the potential roles of each outcome in the SSC are shown on the vertical axis. Additional information includes the number of students undertaking the SSC, and the percentage of students in each cell. This template can be simplified by ‘merging’ the major and minor cells for each outcome (Figure 3), resulting in categorical Yes/No perceptions of outcomes addressed by individual SSCs. Green and red can again be used to colour-code agreement and disagreement respectively; amber is no longer required. Colour versions of these templates applied to the SSCs under study (Figures 4 and 5) can be viewed online using the following web link: http://www.dundee.ac.uk/meded/frames/SSCResearch.html.

Figure 2. Template for examining the role of learning outcomes in individual SSCs. Learning outcomes are numbered 1 to 12 across the horizontal axis. Outcomes are categorized as playing a major role in teaching/assessment, a minor role, or no role at all (N/A). The number of students undertaking the SSC is indicated by n. Individual cells are populated by percentages.

Global comparison (all SSCs)

The second approach involved using the χ² statistic to assess the differences in perception between supervisors and students across the entire Phase 2 SSC programme. This analysis was likewise performed in two ways: first, categorizing individual outcomes as major/minor/not addressed; second, categorizing them as addressed/not addressed (Yes/No in Figure 3).

Figure 3. Simplified template for examining the role of learning outcomes in individual SSCs. Simplified version of Figure 2. Outcomes are categorized as playing a role in teaching/assessment, or not.

Results

All SSC supervisors (n = 69) completed the outcomes template illustrated in Figure 1 (100% response rate). Two hundred and eighty six students completed 648 SSCs. Six hundred and forty four feedback forms were completed (99.4% response rate).

Individual SSCs

Figure 4 illustrates the findings when perceptions were compared by individual SSC, using the template shown in Figure 2. Figure 5 illustrates the same findings using the simplified template shown in Figure 3. Figures 4 and 5 are colour-coded as described above, and it is essential that they are viewed in colour; they can be accessed online at: http://www.dundee.ac.uk/meded/frames/SSCResearch.html.

Global comparison (all SSCs)

Table 1 illustrates the comparison of perceptions across the entire Phase 2 SSC programme, using categorical perceptions (i.e. outcomes perceived as addressed/not addressed by SSCs). The corresponding analysis was also performed with outcomes categorised as major/minor/not addressed (data not shown). Table 1 shows that significantly more students than supervisors thought that outcome 2 was a component of teaching and assessment, whereas more supervisors thought that outcomes 6 and 7 were components of teaching and assessment.

Table 1.  Learning outcomes addressed by SSCs: perceptions of students and supervisors

		Students		Supervisors
Learning outcome	Perception	n	%	n	%	χ^2	p
What a student will be able to do
1. Competent in clinical skills	Addressed	323	50.2	41	59.4	2.14	0.1
	Not addressed	321	49.8	28	40.6
2. Competent to perform practical procedures	Addressed	269	41.8	19	27.5	5.24	0.02*
	Not addressed	375	58.2	50	72.5
3. Competent to investigate a patient	Addressed	402	62.5	41	59.4	0.42	0.5
	Not addressed	241	37.5	29	40.6
4. Competent to manage a patient	Addressed	442	68.6	44	63.8	0.97	0.3
	Not addressed	202	31.4	26	36.2
5. Competent to give advice on health promotion and disease prevention	Addressed	410	63.7	45	65.2	0.01	0.9
	Not addressed	234	36.3	25	34.8
6. Competent in communication skills	Addressed	528	82.1	67	97.1	6.91	0.009*
	Not addressed	115	17.9	4	2.9
7. Competent to retrieve and handle information	Addressed	603	93.7	71	100	4.8	0.02*^†
	Not addressed	41	6.3	0	0
How the student approaches their practice
8. Understanding of social, basic and clinical sciences and underlying principles	Addressed	542	84.2	63	91.3	3.47	0.06
	Not addressed	102	15.8	6	8.7
9. Appropriate attitudes, ethical understanding and legal responsibilities	Addressed	528	81.9	53	76.8	1.11	0.3
	Not addressed	116	18.1	16	23.2
10. Appropriate decision making skills and clinical reasoning and judgement	Addressed	512	79.5	61	88.4	3.11	0.08
	Not addressed	132	20.5	8	11.6
The student as a professional
11. Appreciation of the role of the doctor within the health service	Addressed	538	83.6	54	78.3	1.23	0.3
	Not addressed	106	16.4	15	21.7
12. Aptitude for personal development	Addressed	610	94.6	65	94.2	0.02	0.89^†
	Not addressed	34	5.4	4	5.8

*significantly different perceptions, ^†Fisher's exact test, 1 degree of freedom applied in each case for χ² test

Discussion

Differences between the ‘planned’, ‘taught’ and ‘learnt’ curriculum are well recognised (Lowry 1992), and indeed must be taken into account in developing (Robley et al. 2005a) and applying (Robley et al. 2005b) robust methodology to curriculum mapping. Effective delivery of teaching requires clarity of purpose among staff and students alike. In recognition of this, learning objectives and outcomes are increasingly made explicit in the contemporary undergraduate curriculum. These considerations apply to all parts of the curriculum, but are especially relevant to SSC programmes. In spite of attempts to achieve consensus (Stark et al. 2005) there is often considerable heterogeneity in the way teaching is delivered (Leung 2002; Murdoch-Eaton et al. 2004). In this report we have compared global perceptions of supervisors and students of the role of specific learning outcomes in the teaching and assessment of SSCs, and examined differences between what the supervisors think they are teaching and students’ perceptions of what they learned. We have also devised a visually accessible system of recording agreement/disagreement between the supervisors and students of individual SSCs. A limitation of the study is the difference in time between the completion of templates by supervisors and students. Supervisors completed the outcomes template at the time they offered the SSC–in most cases at the beginning of the academic year–whilst students completed the template shortly after completion. It is possible that simultaneous completion by both parties may have revealed a closer alignment of perception.

Using these two approaches (individual and global comparisons that can identify differences between supervisors’ intentions and student perceptions of what has been learned), we have identified differences in perception across an entire SSC programme, and at the level of individual SSCs. For example, more students than staff thought that outcome 2 (competent to perform practical procedures) was a component of teaching and assessment, whereas more staff than students thought that outcome 6 (competent in communication skills) and outcome 7 (competent to retrieve and handle information) were. These findings may indicate a need for greater clarity with reference to these outcomes. In addition, the ‘traffic-light’ system allowed us to identify individual SSCs which may warrant closer scrutiny. However, red and amber cells (areas of disagreement) must be interpreted with caution; it is essential to know the percentage and number of students disagreeing with the supervisor. An additional point relates to the second ‘amber’ scenario outlined above (where one party thinks an outcome is a minor component of teaching and assessment, and the other does not think it is addressed at all). It could be argued that this should be considered as a categorical rather than partial disagreement, and therefore be colour-coded red rather than amber. This dilemma is resolved by ‘merging’ the major/minor cells as outlined in Figure 3.

Significant disagreement should prompt review of the quality of teaching, SSC content, provision and content of study guides, and/or the number and experience of staff involved in delivering the SSC. In addition, the maturity or experience of students may affect their perception of the role of learning outcomes.

This study is the first to examine differences in perception between supervisors and students of SSC learning outcomes. It is important for several reasons. First, the combination of these two complementary approaches provides a powerful tool to address this important issue, allowing as it does for global and individual comparisons to be drawn. Second, these approaches can also be applied to other outcomes templates, or indeed to assess consensus within and between institutions on the key tasks (Stark et al. 2005) or purpose (Murdoch-Eaton et al. 2004) of SSCs, or indeed other components of the curriculum. Third, the ‘traffic-light’ system we have devised to record agreement/disagreement between supervisors and students of individual SSCs is a readily accessible visual tool that allows all grades of staff to identify at a glance areas of disagreement that may warrant further scrutiny. Finally, the approaches we have used here may have wide implications for quality assurance of medical education.

Acknowledgements

The authors wish to acknowledge the contribution of staff and students at the University of Dundee Medical School without whose co-operation this study would not have been possible. Declaration of Interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this paper.

Additional information

Notes on contributors

Michael J. Murphy

MICHAEL J. MURPHY, BA(Mod), MB BCh BAO, FRCP Edin, FRCPath, is Senior Lecturer in Biochemical Medicine, and SSC Convenor, at the University of Dundee, Scotland, UK.

Rohini De A. Seneviratne

ROHINI DE ALWIS SENEVIRATNE, MD, MBBS, MMEd, FCCP(SL) is Professor in Community Medicine, Faculty of Medicine, University of Colombo, Sri Lanka.

Sean P. Mcaleer

SEAN MCALEER, D Phil, is Senior Lecturer, Centre for Medical Education, University of Dundee.

Olga J. Remers

OLGA J. REMERS, BSc, MSc, is Assessment Administrative Assistant (SSCs) at the University of Dundee Medical School.

Margery H. Davis

MARGERY H. DAVIS, MD, MB ChB, FRCP is Director of the Centre for Medical Education and Professor of Medical Education, University of Dundee.