Can we create an equivalent educational experience on a two campus medical school?

Abstract

Background: Does the university experience and environment on two very different campuses create perceptions of advantage or disadvantage despite delivery of the same curriculum by the same tutors? Perhaps more importantly, do different types of universities result in varying achievements in learning given similar students?

Aims: The Hull York Medical School (HYMS) runs the same curriculum with the same faculty on two very different campuses and randomly allocates most students to one or the other. HYMS therefore offers an exceptional opportunity to investigate these questions about the perceptions of educational environment and actual academic achievement controlling as much as possible for design and delivery of the curriculum.

Method: The Dundee Ready Education Environment Measure (DREEM) was administered to students in Year 1 and 2 at HYMS to assess perceptions of the course. Examination results were collected for the cohorts to compare actual academic performance on written and clinical examinations.

Results: Minimal differences were found in perceptions of educational environment, with those differences found favouring the less ‘prestigious’ institution. Examination results for written and clinical exams over the first 2 years were found not to differ between campuses.

Conclusions: The results of this study indicate that despite perceptions of one university being ‘better’ than another based on public measures such as league tables, the students’ perceptions of educational environment was quite similar and exam performance showed no differences. This suggests that the prestige or ranking of a university based on common measures and perceptions may have far less impact on student learning than careful design and delivery of the curriculum.

Introduction

Hull York Medical School (HYMS) runs an integrated curriculum combining basic science with early clinical experience, half in primary care. HYMS uses a ‘guided discovery’ Problem Based Learning (PBL) approach similar to some other UK schools. However, HYMS is in a somewhat unusual situation in that most students have been randomly allocated to two campuses but they pursue the same course delivered by the same university based tutors.

Students are allocated randomly by ballot within stratified subgroups to balance gender, ethnicity and number of mature students on each campus. Of the students who responded to the survey, there were no significant differences using chi-square measure for the percentage of students on each campus by gender (p = 0.92 for Year 1 and 0.38 for Year 2), ethnicity coded as white or other (p = 0.69 for Year 1 and 0.79 for Year 2) or number of mature students (p = 0.67 for Year 1 and 0.49 for Year 2).

A very small number of students each year are not randomly allocated because personal situations require them to be in Hull or York. However, these numbers are approximately equal in number (six in York and four in Hull for Year 1; four on each campus for the Year 2) and are demographically similar across the campuses. Removing them from the analysis does not change results and so they have been left in the dataset.

Table

Practice points

Folk wisdom is that a prestigious university offers a better education than a less prestigious one, but there is little research on whether educational environment affects performance. Hull York Medical School randomly allocates students to one of two campuses in very different universities with all students receiving the same curriculum, thus giving an opportunity to test some of these assumptions. The students showed remarkably similar judgements of educational environment as based on the DREEM scale, though Hull scored slightly higher on one subscale, Perceptions of Teaching. Academic performance in written exams and OSCEs over the two years that the cohorts remain on campus showed no significant differences. This study indicates that with equivalent delivery of a well-designed curriculum, major differences in type of educational environment show little impact on academic performance for medical students.

In terms of the curriculum, lectures are delivered via video-link to both campuses simultaneously with the lecturers alternating campus so that numbers of ‘live’ lectures are equivalent on each side. Tutors travel to the two campuses for practicals, question and answer sessions, prosection and workshops, ensuring that the curriculum is delivered by the same faculty and with the same content. Students do have different PBL tutors, but these tutors undergo the same staff development, work from the same facilitator notes and attend regular action learning sets to ensure the outcomes of PBL are as uniform as possible. Also, since each group has a different tutor and there are 16 groups in each year, any differences due to PBL tutor would be likely to be distributed across campuses. Thus, HYMS has an exciting opportunity to explore the effects of university environment on student achievement in the early years of the curriculum while controlling the content and personnel involved in delivery.

The two universities, Hull and York, are very different institutions within the UK system. Hull is a large ‘municipal’ university; York is a much smaller research-based institution. York has a collegiate system on campus; Hull has Halls of Residence at a distance. Hull has a fully functioning, very popular, night club; York a vestigial student union. Thus, HYMS is concerned with ensuring that students receive equivalent academic experience on both campuses, despite potentially very different experiences in other aspects of their lives.

This study was attempting to assess whether the university experience and environment does create perceptions of advantage or disadvantage despite the delivery of the curriculum, which undoubtedly some universities would claim. It is possible that the very ‘atmosphere’ of a Cambridge or Harvard might create the perception in the students that their teaching or learning is better than those at less prestigious campuses. What is of more concern to HYMS is whether actual results, as measured by academic performance, are different despite the same programme, perhaps because of social or motivational factors created by the university environment.

Stromquist (2007) found that in a private research university in the US, attempts to increase appeal to students (and to secure research funds) resulted in hiring of prestigious professors and researchers. Is the mere presence of such high flying role models enough to influence students’ perceptions of the learning environment or, perhaps, to motivate them to higher achievement? Or does size matter? It may be that a smaller sized university would allow more interaction with staff or involvement in student societies, both of which have been linked to higher achievement (Astin 1993).

Because of the very different natures of the two universities and our random allocation of students within a single academic programme, HYMS is in a fortuitous position to begin to assess whether these educational environment factors have a noticeable effect on perceptions or academic performance.

Method

To assess perceptions about factors in the educational environment, we administered the Dundee Ready Education Environment Measure (DREEM) to all first and second year HYMS students in April of the 2004–2005 academic year, the second year of HYMS’ existence. The study was approved by the HYMS Research Board and students were notified that the aggregated results would be used for research as well as internal evaluation and that participation was voluntary. The DREEM scale was developed by a Delphi panel of health professional educators from around 20 countries (Roff & McAleer 2001) and is intended to be a relatively ‘culture free’ inventory of the quality of educational environment in health profession courses.

The DREEM scale (Roff et al. 1997) consists of 50 items covering 5 subscales consisting of 7–12 items each: perception of learning, perception of teaching, academic self-perception, perception of atmosphere and social self-perception.

There are some assumptions made by the DREEM scale that were perhaps less appropriate for the HYMS course; for instance, questions about ‘teachers’ and ‘teaching time’. In a problem-based learning course, while there are lectures, most of the time is taken with facilitated group work, self-directed learning, workshops, electronic discussions and patient contact time, all facilitated by different people and not delivered as traditional ‘teaching’, so it makes it difficult for the students to answer a single question such as ‘the teachers are authoritarian’ when they encounter so many different teachers in a week. However, we did advise students to take a general view of the course and to answer according to their opinion about the bulk of the teaching and learning approach encountered. Despite not having been designed for a PBL course, the DREEM scale has been used successfully in medical schools with a variety of curricular approaches, including other PBL schools (Miles & Leinster 2007) and it was felt that the results would be informative and useful.

The DREEM questionnaire was administered via a web form. The students were provided with summary data after the survey. Written exam and Objective Structured Clinical Examination (OSCE) results were gathered for these cohorts in 2004–2005 and 2005–2006, the two years that they were sited on campus before going out to placements around the larger HYMS area in Year 3 of their course, and the campus groups are mixed together.

Results of the DREEM analysis

Response rates for the DREEM scale were Year 1: 126/139 (93%) and Year 2: 90/129 (70%). The campus response rates were very similar to each other with responses from 61 students in Hull and 65 in York in Year 1, 43 in Hull and 47 in York for Year 2. Missing values for individual items on the 50 item DREEM inventory were replaced by the median response for the question over the whole year cohort. There were very few missing items: Year 1 had 33 individual missing items, or 0.5% and Year 2 had 26, or 0.6%. No one question item had any more than two missing values in Year 2, three had three missing in Year 1 and two had four missing in Year 1.

The score on the DREEM combining all students within each year was 144.4 for Year 1 and 141.6 for Year 2, with the maximum possible score being 200. This difference was not significant (t = 1.27, p = 0.21). Therefore, the scores from the two cohorts were combined for further campus analysis.

Combining the years, the overall DREEM scores for the two campuses were 145.2 for Hull and 141.4 for York. A multivariate ANOVA using the five subscales showed a significant difference (F = 2.45, p = 0.035). Comparisons of the sub-scales adjusted for repeated testing indicated that there were no significant campus differences for the four scales of academic self-perception, social self-perception, perceptions of teachers or perceptions of atmosphere. There was a campus difference on one scale with Hull students rating significantly higher on perceptions of learning (p = 0.009). Table 1 presents the scores for these subscales for the two campuses.

Table 1.  Overall DREEM scores by campus and post-hoc comparisons using Scheffe's tests

Campus	Perception of learning (max 48)	Perception of teachers (max 44)	Academic self-perception (max 32)	Perception of atmosphere (max 48)	Social self-perception (max 28)
Combined	34.2	32.5	21.4	35.5	19.7
Hull (N = 104)	35.0	33.0	21.3	35.9	19.9
York (N = 112)	33.4	32.0	21.4	35.1	19.5
Significance	p = 0.009*	p = 0.07	p = 0.84	p = 0.18	p = 0.28

Results of the examinations analysis

While it is gratifying to find that student perceptions of the educational environment are only marginally different and HYMS is scoring highly in general, there remains the important question of whether academic performance indicates differences between the campuses. We collected the Year 1 and Year 2 written exam and OSCE scores for these two cohorts. We chose to use only these two years since from Year 3, the students are mingled together in groups and are in placements off-campus across North and East Yorkshire and North Lincolnshire.

Scores were collected for the main end of year written exam covering primarily basic sciences such as anatomy, physiology, pharmacology, behavioural science and clinical sciences. We also collected the scores on the OSCE. For both cohorts (2003–2008 and 2004–2009) exam scores were collected for Year 1 and Year 2. There were no significant differences as measured by the t-test between the campuses on written exam results in Year 1 (61% York; 60% in Hull, p = 0.54) or Year 2 (66% in York; 65% in Hull, p = 0.28) nor on the OSCE scores in Year 1 (73% in York; 74% in Hull, p = 0.33) or Year 2 (72% on both campuses, p = 0.83).

Conclusions

Overall, there was very little difference in perceptions of educational environment across the two universities as measured by the DREEM scale. The reasons for the one campus difference that was found, with York scoring lower than Hull on ‘perceptions of teaching’, are purely speculative, but interesting in the HYMS context because public perception is that York is a more desirable place to live than Hull (Jordison & Kieran 2004) and that the University of York is a ‘better’ university, at least according to most league tables. Also, there were more difficulties with recruiting of administrative staff and more disruptions in the physical infrastructure at Hull at that time with extensive building works and network problems. It could be that a reaction to negative publicity or the common experience of hardship has created a more cohesive and supportive student cohort in Hull who rate their learning experience more highly, but this remains only a hypothesis to be explored.

It is also interesting that HYMS has received a higher score on DREEM than most other schools’ results that we have found reported in the literature. These scores range from 78 for Year 3 Canadian chiropractic students to 139 for Dundee (Roff 2005). Some other scores reported (all from Roff (2005) unless indicated otherwise) are 102 and 107 for schools in Saudi Arabia, 100 in Yemen, 127 in Oman, 134 in the Netherlands, 125 for a Malay dental technology programme, 118 in China, 110 in the West Indies (Bassaw et al. 2003), 118 in Nigeria and 130 in Nepal (Roff et al. 2001), 124 for a ‘large UK medical school’ (Dunne et al. 2006) and 125 for a Malaysian dental school (Ibrahim 2008).

While it is speculation as to why HYMS scored so highly, it would be interesting to explore further whether PBL based, student-centred curriculae score particularly well, as suggested in Roff (2005), or whether it is a ‘new school’ phenomenon, or something else entirely. The only study we have found where scores were similar to HYMS was from the University of East Anglia Medical School with a score of 143 for students at the end of their first year which was also the first year of UEA's medical school (Miles & Leinster 2007). Again, this is both a new medical school and a PBL school. The fact that two years of students at HYMS scored relatively highly on DREEM, however, might indicate that it is the nature of a PBL course rather than the effect of being the first cohort that is reflected in the scores.

Although lack of significant difference between the campuses on examinations does not imply that the experience is identical, it seems that non-academic environmental factors may have a relatively small effect on academic achievement, or at least that campus strengths and weaknesses may cancel each other out, despite folk wisdom that a prestigious university equals a better education. The UK's National Union of Students vice-president Wes Streeting, quoted in a BBC report (BBC 2008), said a recent report on league tables proved what the union had long argued: ‘League tables can only advise prospective students on the prestige associated with a university, as opposed to how well it might suit their own individual needs.’

It is intriguing that teaching environment ratings showed the only significant difference, given that students at HYMS have the same curriculum and the same lecturers. It may be that students’ perceptions of their teaching and learning environment may not have a great impact on their actual learning achievements, though this is an area in need of further research since few differences in perceptions were found in this study.

It may be that the important differences for learners between Oxbridge and the University of Poppleton are not in their public reputation or the number of Nobel laureates wafting across campus, but in interaction with tutors, a clearly constructed curriculum, positive role models, and the relevance of the course to learners, all of which are theoretically achievable at any institution and are often considered the important factors of a good educational environment (Hutchinson 2003). With differential fees looming in the UK, students may be well advised to find out more about these factors rather than looking to league tables, media portrayals and glossy prospectuses to find the best place for a good educational outcome.

Acknowledgement

Portions of this study were presented at ASME 2005.

Declaration of interest: The author reports no conflicts of interest. The author alone is responsible for the content and writing of the paper.

Additional information

Notes on contributors

Jean McKendree

Dr JEAN MCKENDREE is Senior Lecturer in Medical Education at the Hull York Medical School and has a background in cognitive science. Her research involves applying cognitive theory to education in areas including educational technology, the role of discussion in learning, graphical representations for reasoning, metacognition and critical thinking skills.