Perceptions by medical students of their educational environment for obstetrics and gynaecology in metropolitan and rural teaching sites

Abstract

Background: Medical student education in Western Australia is expanding to secondary level metropolitan hospitals and rural sites to accommodate workforce demands and increasing medical student numbers.

Aims: To determine if students’ perceptions of the teaching environment for obstetrics and gynaecology differ between tertiary, secondary level metropolitan hospitals and rural sites, and to determine if students’ perceptions of their learning environment are associated with improved academic performance.

Method: An evaluation was conducted of medical students’ perceptions of their learning environment during an obstetrics and gynaecology program at a variety of sites across metropolitan and rural Western Australia. The evaluation was based on the Dundee Ready Education Environmental Measure (DREEM) questionnaire.

Results: There were no significant differences in students’ perceptions of their learning environment between the tertiary hospital, combined programs involving a tertiary and secondary metropolitan hospital, rural sites with a population of more than 25,000 and rural sites with a population less than 25,000 people. Perceptions were similar in male and female students. The overall mean score for all perceptions of the learning environment in obstetrics and gynaecology were in the range considered to be favorable. Higher scores of perceptions of the learning environment were associated positively with the measures of academic achievement in the clinical, but not written, examination.

Conclusion: Medical students’ perceptions of their learning environment in obstetrics and gynaecology were not influenced by the geographical site of delivery or their gender but were positively related to higher academic achievement. Providing appropriate academic and clinical support systems have been put in place the education of medical students can be extended outside major hospitals and into outer metropolitan and rural communities without any apparent reduction in perceptions of the quality of their learning environment.

Introduction

Evaluation of educational programs is critical for learning to be most effective (Goldie 2006). The evaluation process provides an opportunity to identify and address any areas in which improvements may be made and to identify those aspects that reflect effective educational practice (Genn 2001b). The perceptions held by students of their educational environment are integral to their experience, rendering it vital that this aspect is also evaluated. It has been suggested that students perceive the interrelationships between the curriculum, their learning environment, and the quality of the program as their education climate, and this climate is the spiritual essence of the school and its curriculum (Genn, 2001b). The perceptions held by students of their learning climate have been shown to influence their satisfaction, academic outcomes and learning behaviors, including their approach to study and assessment (Genn 2001a; Lizzio et al. 2002; Nijhuis et al. 2007). In our current era of expanding medical education, it is important that we evaluate the effectiveness of our innovations together with the established modes of curriculum delivery.

Australia is increasing the number of its doctors in training. These increased numbers, coupled with other changes in health care management, have created a demand for new approaches to medical education (Olson et al. 2005; Joyce et al. 2007). Much of our previous reliance on education within central and tertiary level hospitals has given way to the commissioning of secondary level hospitals, and rural and community-based centres as sites of learning (Thistlethwaite et al. 2007). These changes provide both challenges and opportunities in meeting education and curriculum requirements.

The teaching of medical students in Western Australia has undergone major changes in recent years. This state geographically is the largest in Australia, with a population of 2 million people over an area of 2,252,500 km², of whom 1.5 million reside in Perth, the capital city. Most referral level medical resources, and all five of the major teaching hospitals, are within the capital city resulting in centralization of resources. The medical system has relied on local delivery of primary and some secondary level health care, but transfer of patients with more major problems to Perth, often involving referral distances up to 2000 km or more. In recent years teaching sites have been opened in peripheral areas of the capital city and in rural areas. The impetus for this expansion has been to improve rural workforce recruitment and retention, and more recently, to provide opportunities for increasing numbers of medical students.

This study has evaluated the effect of location of education delivery across a spectrum of metropolitan and rural sites by measuring students’ perceptions of their learning climate. The study centred on the obstetrics and gynaecology unit, delivered by the School of Women's and Infants’ Health at The University of Western Australia (UWA). The obstetrics and gynaecology unit is taught in year five of a 6-year undergraduate medical curriculum. At the time of this study, the unit was taught at 12 sites in rural and metropolitan Western Australia. Previously, this unit was located almost exclusively at the sole tertiary level women's hospital in central Perth. Briefly, the delivery of the unit in the metropolitan area involves students undertaking a 10-week term including an extensive orientation week; an individual clinical roster; a structured academic timetable featuring workshops, lectures and weekly small group tutorials; and finishing with a planned week of review fora and student presentations.

Obstetrics and gynaecology is also taught in the Rural Clinical School (RCS). The RCS was launched in 2003 undertaking to teach the year 5 curriculum to approximately one-quarter of the students enrolled in year 5 of the course. At the time of the study, the RCS encompassed 10 teaching sites throughout rural and remote Western Australia (Figure 1). The RCS program is based on 1 year of learning by a student at a single site, with the entire year's curriculum spread over the year in a continuous format. The teachers include specific RCS staff and local hospital and visiting clinicians. Rural sites vary in distance from the capital city ranging from 180 to 2380 kilometres and include local hospitals and community centres. The majority of the teaching is conducted by general practitioner medical officers with specialist obstetrician/gynaecologists at half the sites. Allocation of the number of students for a particular site is determined by the size of the teaching centre and ranged from 3 to 10.

Figure 1. Map of Western Australia showing the site of the capital city, Perth, and the 10 rural locations at which medical students may be placed for teaching in obstetrics and gynaecology.

The purpose of this study was to determine if students’ perceptions of their learning environment in obstetrics and gynaecology differed according to site of delivery of the unit. We also aimed to determine if students’ perceptions of their learning environment were associated with their academic scores measured at the end-of-year examination. The students’ perceptions of their learning environment were measured by the Dundee Ready Education Environment Measure (DREEM) Questionnaire. The DREEM questionnaire is a validated generic tool (Roff 2005) that has been used for many purposes that have included creating a picture of the strengths and weakness of a teaching program (Till 2004; Avalos et al. 2007); comparing the outcomes of a program delivered at different centres (Varma et al. 2005); comparing academic achievers and under-achievers (Mayya & Roff 2004); and evaluating educational environment to academic achievement in a nursing school (Pimaryon et al. 2000).

Table

Practice points

DREEM was a valuable and readily administrable tool to measure student perceptions of their learning environments over a variety of teaching sites Obstetrics and gynaecology can be taught across a range of metropolitan and rural sites without affecting student perceptions of their learning environment, providing appropriate clinical and academic support is in place Student perceptions of their learning environment are linked to their clinical academic outcomes

Materials and methods

Approval to conduct the study was provided by the Ethics Committee of The UWA. An information sheet about the study, including an assurance that participation would not impact upon their individual academic outcomes, was given to all students prior to administering the questionnaire. Students were free to choose to not participate.

The DREEM questionnaire is based on a Likert Scale and comprises 50 items that allot a global score out of 200. DREEM assesses five categories of students’ perceptions: perceptions of learning, perceptions of teachers, academic self-perceptions, perceptions of atmosphere, and students’ social self-perception.

The study included 172 students, of whom 131 were based in the metropolitan region and 41 in rural sites. Each student was asked to complete the questionnaire with relation to their learning specifically in obstetrics and gynaecology. Students were also asked to identify themselves by name to enable their end-of-year results to be linked to their perception of the learning environment. All completed questionnaires were placed in a blank envelope by the student and handed to an administrative assistant.

In the metropolitan region, each group of students over the four terms in the academic year was recruited in the final week of their 10-week term. In the rural sites, all students were asked to contribute to the study 8–10 weeks prior to the conclusion of the academic year.

In order to protect the identity of individual students, and to counter the small numbers in many sites, the rural participants were allocated to two groups based on whether the population in their region was more or less than 25,000. Students in the metropolitan regions were also assigned to two groups: the first being students who were placed only at the tertiary hospital and the second included those students who were allocated to a 4-week placement at a secondary metropolitan hospital, with the remainder of their term at the tertiary hospital.

The academic outcome for each student was calculated by combining the percentage mark for the written exam, consisting of modified essay and short answer questions, and the total of the four obstetrics and gynaecology stations in the year 5 integrated Objective Structured Clinical Examination. Both examinations were conducted at the end of the academic year and were taken in an identical manner by students from rural and metropolitan sites. Assessors were recruited from both metropolitan and rural units, and all were blinded to the educational sites of the students. The marking guides for both examinations were subject to structured standardization processes and each assessor agreed formally on the marking criteria.

In the statistical analyses, continuous data were summarized using means and standard deviations. Categorical data were summarized using frequency distributions. Some continuous outcomes (obstetric and gynaecology academic outcomes) were allocated to binary groups using a median cut-off to enable presentation of data. Independent samples t-tests for equality of means and one-way analyses of variance were used to determine differences between groups. Linear regression was performed on raw obstetric and gynaecological academic outcomes, with simultaneous adjustment for Tertiary Entrance Rank (TER), separately modeling DREEM total score and subscale scores as individual predictors. The TER score is the mark given on completion of secondary school to determine eligibility to enter university and was available for 74% of the students in the study. Coefficient of determination (R²) was used to measure what proportion of variance was explained by the predictors used in the model. A p-value of <0.05 was set as statistically significant. SPSS (version 15.0: SPSS Inc. Chicago, Illinois) statistical software was used for data analysis.

A post hoc power calculation demonstrated that the numbers in the four comparison groups provided statistical power of at least 90% to detect a difference of 10 points in the DREEM score which has a maximum score of 200 (α = 0.05).

Results

The DREEM questionnaire was completed by 94% of the students. Of the students in this cohort, 52% were placed exclusively at the major tertiary hospital; 29% were allocated to the tertiary hospital with a 4-week placement at a secondary metropolitan hospital; 11% were placed at one of seven rural sites with a population of less than 25,000 people; and the remaining 9% were at one of three rural sites with a population of more than 25,000 (Table 1).

Table 1.  Demographics of medical students who participated in the study

Site of teaching / clinical placement	All	Male	Female	Overall p-value
Tertiary hospital (only)	84 (52%)	41 (49%)	43 (51%)	0.548
Tertiary hospital with placement at a secondary metropolitan hospital (4 weeks)	46 (29%)	21 (46%)	25 (54%)
Rural clinical school sites < 25000 population	17 (11%)	7 (41%)	10 (59%)
Rural clinical school sites > 25000 population	14 (9%)	4 (29%)	10 (71%)

Data represented as N (%)

Of the total number of students, 55% were female and 45% were male (p = 0.548) (Table 1). There was a greater percentage of female medical students (65%) at the rural sites, although this difference was not significant (p = 0.465).

The overall mean score for the DREEM questionnaire was 149 ± 18.2 (SD), which is 75% of the maximum possible score (Table 2). There were no significant differences in scores for each of the five domains of the DREEM questionnaire or in responses of the two genders.

Table 2.  DREEM score by gender

	All	Male	Female	p-value*
Total score	149 ± 18.2	147.7 ± 18.6	150.1 ± 18.0	0.347
Maximum = 200
Perceptions of learning	36.6 ± 5.5	35.9 ± 5.6	37.3 ± 5.3	0.229
Maximum = 48
Perceptions of teaching	34.4 ± 4.3	34.6 ± 4.3	35 ± 4.3	0.823
Maximum = 44
Academic self-perceptions	21.8 ± 3.6	21.6 ± 4.0	22 ± 3.3	0.525
Maximum = 32
Perceptions of atmosphere	36 ± 5.2	35.9 ± 4.9	36.1 ± 5.5	0.741
Maximum = 48
Social self-perceptions	19.8 ± 3.2	19.6 ± 3.5	19.9 ± 3.5	0.679
Maximum = 28

Data represented as mean ± SD.

Maximum is the maximum possible score that can be attained in the DREEM questionnaire domains.

*One-way ANOVA.

The overall DREEM scores and scores within each domain by geographical site are shown in Table 3. There were no statistically significant differences in the DREEM domain scores according to site of teaching. Students’ academic self-perception and social self-perception scores tended to be lower across all teaching sites when compared with perceptions of learning, atmosphere and teaching but the differences were not significant. In all sites, the perception held by students of their teaching scored the highest percentage of maximum score when compared with the other domains. The rural sites with a population of fewer than 25,000 scored marginally lower in all domains of the DREEM score, although the differences were not statistically significant.

Table 3.  DREEM Scores by geographical teaching sites

	Tertiary hospital	Tertiary/ second. hospital^a	Rural sites (small)^b	Rural sites (large)^c	p-value*
Total score	148.8 ± 18.1	151.9 ± 16.6	141.9 ± 18.9	148.9 ± 22.8	0.291
Maximum = 200	(74%)	(76%)	(71%)	(75%)
Perceptions of learning	36.7 ± 5.2	37.2 ± 5.0	34.4 ± 7.0	36.9 ± 6.4	0.334
Maximum = 48	(77%)	(78%)	(72%)	(77%)
Perceptions of teaching	34.7 ± 4.3	35 ± 4.1	34.2 ± 4.7	35.6 ± 4.2	0.819
Maximum = 44	(79%)	(80%)	(78%)	(81%)
Academic self-perceptions	21.6 ± 3.6	22.9 ± 2.8	20.5 ± 4.0	21.1 ± 4.9	0.071
Maximum = 32	(69%)	(72%)	(64%)	(66%)
Perceptions of atmosphere	36.3 ± 4.7	36.8 ± 4.7	33.2 ± 4.4	35.2 ± 8.8	0.085
Maximum = 48	(76%)	(76%)	(69%)	(73%)
Social self-perceptions	19.5 ± 3.3	20.2 ± 3.2	19.7 ± 3.3	20.1 ± 2.9	0.714
Maximum = 28	(70%)	(72%)	(70%)	(72%)

Data represented as mean ± SD. The mean in also represented as 9% of the maximum score.

Maximum is the maximum possible score that can be attained in the DREEM questionnaire domains.

*One-way ANOVA.

^aTertiary hospital with placement at a secondary metropolitan hospital (4 weeks).

^bRural clinical school sites < 25000 population.

^cRural clinical school sites > 25000 population.

The total DREEM scores together with individual scores for each of the five domains according to results of the students’ end-of-year examinations are shown in Table 4. Mean marks for the examination have been categorized as low or high (binarised on median mark) to enable presentation of data. There were no statistically significant associations between total DREEM scores, or the five individual domains, and results of the written examination. Results of the clinical examination, however, were significantly associated with the total DREEM score and each of the five domains. The strongest associations between DREEM scores and total examination scores were observed for social self-perceptions (p < 0.001), perceptions of learning (p = 0.017) and academic self-perceptions (p = 0.024).

Table 4.  DREEM scores by binarised^a obstetrics and gynaecology examination results (written examination, clinical total marks)

	Written examination			Clinical examination			Total mark
	Low^b	High^c	p-value	Low^b	High^c	p-value	Low^b	High^c	p value*
Total score	146.8 ± 18.1	151 ± 18.2	0.142	144.1 ± 18.3	153.6 ± 17.1	0.001	144.7 ± 18.7	152.7 ± 17.5	0.006
Maximum = 200
Perceptions of learning	35.9 ± 5.9	37.3 ± 5.0	0.127	35.1 ± 5.7	38 ± 4.9	0.001	35.5 ± 5.9	37.6 ± 4.9	0.017
Maximum = 48
Perceptions of teaching	34.5 ± 4.6	35.1 ± 3.9	0.359	34.1 ± 4.5	38 ± 4.9	0.034	34.1 ± 4.7	35.3 ± 3.6	0.091
Maximum = 44
Academic self-perceptions	21.5 ± 3.8	22.1 ± 3.4	0.320	21 ± 3.6	22.6 ± 3.4	0.004	21.1 ± 3.7	22.4 ± 3.5	0.024
Maximum = 32
Perceptions of atmosphere	35.4 ± 4.8	36.6 ± 5.6	0.174	35 ± 4.8	36.9 ± 5.4	0.023	35.1 ± 4.6	36.8 ± 5.6	0.034
Maximum = 48
Social self-perceptions	19.5 ± 3.4	20.1 ± 3.1	0.238	18.9 ± 3.3	20.6 ± 2.9	0.001	18.9 ± 3.5	20.6 ± 2.9	0.001
Maximum = 28

Data represented as mean ± SD.

^abinarised on median.

^bLow mark < median; ^cHigh mark ≥ median.

*Independent samples t-test.

When further evaluated by linear regression analyses (Table 5) there were no statistically significant associations between DREEM total or individual domain scores and the written examination result except for perceptions of learning (p = 0.025). Results of the clinical examination, however, were significantly associated with the total DREEM score (p = 0.003) and four of the domains (perceptions of learning p = 0.006; perceptions of teaching p = 0.034; perceptions of atmosphere p = 0.006; and social self-perception p = 0.007). Total examination results were significantly associated with the total DREEM score (p = 0.006) and three of the domains (perceptions of learning p = 0.003; perceptions of atmosphere p = 0.012; and social self-perception p = 0.009). Academic self-perception was not significantly associated with any of the academic outcomes. The TER result in the linear regression model, however, explained only a small proportion of the variance ranging from <1% to 7%.

Table 5.  Linear regression showing prediction of 5th year obstetrics and gynaecology examination marks using DREEM scores^a

	Written examination		Clinical examination		Total marks
	R^2*	p-value**	R^2*	p-value**	R^2*	p-value**
DREEM score	0.027	0.069	0.070	0.003	0.060	0.006
Perception of learning	0.041	0.025	0.061	0.006	0.068	0.003
Perception of teaching	0.010	0.273	0.037	0.034	0.027	0.068
Academic self-perception	0.002	0.609	0.025	0.079	0.013	0.215
Perception of atmosphere	0.021	0.108	0.062	0.006	0.050	0.012
Social self-perception	0.027	0.069	0.059	0.007	0.055	0.009

^aadjusting for TER mark.

*R² represents proportion of variation in the model explained by the TER.

**p-value of predictor.

Eleven students did not participate in the study. All non-responders were female; nine of these students were from the rural sites and two were from metropolitan placements. A comparison of examination scores achieved by non-responders and responders showed no statistically significant differences in any of the measured categories.

In the rural sites, total DREEM scores and scores within each of the five domains were not influenced by the presence or absence of a resident specialist obstetrician/gynaecologist (p = 0.149).

Comments

The results of the present study indicated that students’ perceptions of their educational environment in obstetrics and gynaecology were not influenced by the geographical site of their program or their gender. Each of the sites in the present study differed in terms of patient case load and teaching staff, with the program at each site adapted to local opportunities. Our findings provide strong evidence that undergraduate teaching in obstetrics and gynaecology can be expanded beyond the confines of major metropolitan hospitals to include outer metropolitan and rural centres. The findings from our study are in contrast to the results of a previous study conducted in Sydney, in which students rated the rural clinical experience more highly than the metropolitan setting (Lyon et al. 2008). In that study, students described their rural experience as superior in terms of support provided in the clinical setting, opportunities to develop their clinical skills, the approach of their teachers, and their own confidence and sense of self-efficacy (Nijhuis et al. 2007; Lyon et al. 2008).

The overall mean DREEM score (149 ± 18) for the teaching environment in obstetrics and gynaecology in the present study compared favorably with scores in other studies (Pimaryon et al. 2000; Al-Hazimi et al. 2004a; 2004b; Varma et al. 2005; Avalos et al. 2007). Further, the results within each of the domains were similar at the various sites indicating that the students’ responses were generally positive regardless of location. This consistency may have resulted at least in part from the design of the overall program. The obstetrics and gynaecology curriculum is primarily developed and maintained at the central metropolitan site. This curriculum contains clearly outlined clinical and academic objectives and aims to produce graduates who have the essential core knowledge, skills and attributes to provide appropriate and safe care for women from diverse backgrounds. Each year, updated problem and case-based learning tutorials are made available to educators at each site, and lectures are video-conferenced from the main campus to the rural sites. All sites have a dedicated teaching coordinator and the curriculum and its delivery are subject to ongoing collaboration between educators at all locations. Each site is responsible for monitoring its program delivery and fostering a culture of responsive evaluation to ensure an optimal curriculum for their environment. The program taught at each site, however, remains closely aligned with the formal curriculum.

The site of education, the curriculum and the coordinators all play vital roles in producing an effective learning environment but the factors influencing student experience are many. In addition to data such as those delivered by the DREEM questionnaire, a comprehensive evaluation may require qualitative work with the students to explore their experiences. This combined approach has recently been reported from another medical school (Whittle et al. 2007). Evaluation of this type is not only of value for remedial purposes, but also assesses those experiences that lead students to rate their experience as positive. It may also be of value to separate issues concerned with academic learning and experiences in the clinical environment as these scenarios may provide different challenges (Seabrook 2004). Delivery of a curriculum in the clinical environment is not a contained entity but is a comprehensive experience of all that the student learns and is exposed to, the physical environment, the teaching staff, as well as their peers, patients and families (Genn 2001b).

The findings of this study revealed that students’ perceptions of their learning environment in obstetrics and gynaecology were positively associated with their academic outcomes for the unit. Those students who perceived their learning environment positively were more likely to score above the median mark in their clinical examinations and overall mark for the program. For the clinical examination, positive associations were observed between academic achievement and the domains reflecting perceptions of learning, perceptions of teaching, perceptions of atmosphere and social self-perception. This finding supports those from previous studies conducted in other disciplines suggesting that students’ perceptions of their learning environment have a positive influence on their academic outcomes (Lizzio et al. 2002; Mayya & Roff 2004).

Students’ perceptions of their learning environment, however, were not associated with academic achievement in the written examination. The reason for the discrepancy in the associations between students’ perceptions of their educational programs and the results of the written and clinical exam is uncertain. It may at least in part result from the fact that most medical students have well developed written exam techniques by the 5th year of university studies, whereas development of the clinical skills required for the clinical skills examination are concentrated in the later years of their course. Those students who have a positive experience of the learning environment may take better advantage of clinical experiences, leading to higher marks in their clinical skills examination.

In summary, the results of this study support the expansion of medical student education to outer metropolitan and rural centres. The traditional reliance on teaching within major central hospitals need not continue and medical schools can embrace teaching opportunities at more distant locations with confidence. Such expansion, however, requires academic and clinical support, and is likely to benefit from ongoing measurement of the effectiveness of the program. This measurement should include perceptions of the program by the students themselves. The DREEM questionnaire provides an effective method by which this evaluation may be performed. Positive associations observed between the students’ perceptions of their experience and the results of the clinical examination, but not the written examination, suggest that factors underpinning academic success in the early years of the course may differ from those operating in the years when clinical experiences become central to success.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Dianne F. Carmody

DIANNE CARMODY initiated the study and was the prime author of the manuscript. She is a course coordinator and associate professor at the School of Women's and Infants' Health, The University of Western Australia.

Angela Jacques

ANGELA JACQUES conducted the statistical analysis and approved the final manuscript. She is a biostatistician at the Women and Infants’ Research Foundation based at King Edward Memorial Hospital, Western Australia.

Harriet Denz-Penhey

HARRIET DENZ-PENHEY contributed to data collection, analysis and approved the final manuscript. She is a Senior Research Fellow at the Rural Clinical School of Western Australia, The University of Western Australia.

Ian Puddey

IAN PUDDEY was involved in conception, analysis and approved the final manuscript of the study. He is the Dean of Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia.

John P. Newnham

JOHN P NEWNHAM was involved in conception, analysis and approved the final manuscript of the study. He is a Professor of Obstetrics and Gynaecology (Maternal and Fetal Medicine) and the Head of the School of Women's and Infants' Health, The University of Western Australia.