Workplace learning in general practice: Supervision, patient mix and independence emerge from the black box once again

Abstract

Background: Medical students increasingly participate in rotations in primary care settings, such as general practices. Although general practices can differ markedly from hospital settings, research on the instructional quality of general practice rotations is relatively scarce.

Aim: We conducted a study to identify which aspects make a significant contribution to the instructional quality of general practice clerkships, as perceived by students.

Method: After completing their general practice attachment, 155 fifth-year medical students filled out an evaluation questionnaire. Exploratory factor analysis and reliability analysis identified clusters of correlated independent variables. Subsequent regression analysis revealed the relations between the reduced set of independent variables and the dependent variable ‘Instructional quality’.

Results: Both the quality of supervision and the size and diversity of the patient mix substantially affected students’ experienced instructional quality. Opportunities and facilities to perform independently were correlated with instructional quality, but did not affect the instructiveness directly.

Conclusions: Supervision, patient mix and independence are crucial factors for learning in general practice. This is consistent with findings in hospital settings. The perceived quality of instruction hinges on supervision, which is not only the variable most strongly related to instructional quality, but which also affects both the patient mix and students’ independence.

Introduction

Traditional apprenticeships were once the main or even single component of learning medicine. Basically, the master taught his apprentice ‘how to heal’. Although the master apprenticeship model is still clearly visible in medical curricula, in contemporary medical education the concept of workplace learning has changed considerably.

Nowadays, medical students are expected to learn to become doctors who ‘master complex theory and integrate it into their practice’ (Dornan 2006). For this purpose, students need to integrate biomedical knowledge, practical skills and a professional attitude and they will have to translate these competencies into competent behaviour. The integration and translation of competencies is best mastered in authentic and context-related learning environments, such as clinical clerkships (Dolmans et al. 2002a; Daelmans et al. 2004; Dornan 2006). It is of vital importance to ensure that such authentic learning settings enable students to learn instead of ‘merely demonstrating’ readily or recently acquired knowledge and skills. Students should be empowered to build and expand their knowledge and skills for clinical reasoning and decision making in an environment that offers fertile ground for personal and professional development. At least, those are the objectives of workplace learning from an educational perspective.

Clerkship settings are melting pots of people with different roles (students, trainee doctors, registrars, consultants, patients and nurses, etc.) and different priorities (delivery of high quality care, getting well, making a profit and getting a good education). It is therefore not surprising that the literature on workplace learning shows that the instructional quality of clerkships is determined by at least as many factors as there are people and priorities that constitute the learning environment of the workplace.

Literature on workplace-based training in medicine is dominated by hospital-based studies. Factors that have been shown to determine the instructional quality of clerkships include: workplace atmosphere (Stark 2003; van der Hem-Stokroos et al. 2003; Dornan 2006), number and quality of learning opportunities (Dolmans et al. 2001; van der Hem-Stokroos et al. 2003), patient mix (Dolmans et al. 2002a) and attributes of students as well as teachers (Stark 2003; Dornan 2006), the latter including the quality of supervision (Dolmans et al. 2002a), observation, feedback and assessment (van der Hem-Stokroos et al. 2003; Daelmans et al. 2004). Together these factors contribute to a further critical component of workplace learning: (supported) participation (van der Hem-Stokroos et al. 2003; Daelmans et al. 2004; Dornan et al. 2007). It should be noted that studies have revealed not only which factors are important for workplace learning, but also that the quality of these factors often leaves a great deal to be desired (Dolmans et al. 2001; van der Hem-Stokroos et al. 2001; Stark 2003; Daelmans et al. 2004).

The instructional quality of general practice settings has received less attention from researchers. At the same time such studies are increasingly needed, as primary care comes more and more to the forefront in health care systems and medical education (Shipengrover & James 1999; Gordon et al. 2000). A general practice makes different demands on students and teachers than other clinical settings (Shipengrover & James 1999), supposedly because of differences in organisational size, structure, culture, patient population, learning opportunities and goals and tasks in the delivery of health care. In other words, general practice attachments are melting pots, just like hospital workplaces, only somewhat smaller and containing a different mix of people and priorities.

Although, we know little about determinants of the instructional quality in general practice settings, we do know something about differences between education in general practice and in hospital workplaces. General practice clerkships are rated by students as significantly more enjoyable than hospital clerkships and students say that learning objectives for history taking and physical examination are easier to attain in general practice (Murray et al. 1999). Hospitals, on the other hand, offer more opportunities to learn about performing procedures (Kurth et al. 2001) and disease management, and are more helpful in learning to write history, examination and progress notes (Murray et al. 1999).

Student–teacher ratios are another noticeable difference between primary care and hospitals. The lower ratio in general practice attachments allows more emphasis on the interpersonal interaction between teacher and student, fostering a relationship that facilitates feedback and rapport (Leone-Perkins et al. 1999). Probably related to this, according to Silverstone's qualitative research, the general practitioner (GP) teacher is an important determinant of the perceived effectiveness of a community attachment. This may explain why students have been known to label a successful attachment simply ‘A good GP’. A ‘good GP’ not only embodies the GP as a good teacher and role model but also a positive learning climate. Thus, a pattern in which learning opportunities, personal interaction and role modelling are intertwining appears to determine the perceived quality of learning in primary care (Silverstone et al. 2001).

In the face of the increasing prominence of primary care settings in medical education, the different organisational structures of primary care and hospital workplaces and the relative paucity of research into primary care as a learning environment (Gordon et al. 2000), our objective was to bring to the surface the instructional components that are central to the quality of undergraduate training in general practice settings. The identification of these key factors could lead primarily towards improvement of workplace learning, and could guide teacher development as well. For this purpose, we sought to answer the following question: which factors have the strongest impact on the instructional quality of clerkships in general practices as experienced by medical students? Additionally, we compared the answers to this question with findings from studies in hospital settings.

Table

Practice points

Students judge the general practice clerkship's instructional quality positively. Supervision quality, patient mix and students’ independence relate to perceived instructional quality. General practice supervisors can provide vital ingredients for the clerkship's effectiveness. Determinants of instructional quality in general practice seem comparable to those in hospitals, although stakeholders’ interpretation of these key learning ingredients needs further elaboration.

Methods

Setting

This study was conducted at the Faculty of Health, Medicine and Life sciences (FHML) of Maastricht University in 2009. During the obligatory 10-week general practice clerkship in year five of the 6-year undergraduate medical curriculum, students are attached to a general practice for 3.5 days a week. Students usually have one GP preceptor whom they observe and who observes them during patient consultations. The remaining 1.5 days of the work week are taken up by a 1-day tutorial group session and a half day designated for selfstudy.

Instrument

We used data from the regular clerkship evaluation questionnaire, which students fill out anonymously after completion of each clinical attachment. Ethical approval for conducting this study was not required. In this study, we looked at a subset of 17 questionnaire items relating to aspects that are likely to impact on learning in general practice. The items were rated on a 5- or 10-point Likert scale, depending on the question. ‘Instructional quality’ was the dependent variable and the other 16 items were the independent variables (Table 1).

Table 1.  Description of items and variables, original rating scales and eigenvalues, and explained variance after explanatory factor analysis

		Item description	Original rating scale (points)	Eigenvalue	Explained variance (%)	Cronbach's alpha
Dependent variable		Instructional quality	10	–	–	–
Independent variables	Supervision	Accessibility of supervisor when students wished to ask questions or have things explained to them	5	6.863	42.891	0.876
		Supervisor explains what is done and why	5
		Observation of students during patient consultations	5
		Feedback after observation	5
		Time spent on supervision	5
		Working climate	10
		Organisation	10
		Supervision	10
		Average number of patient discussions per week	5
		Average number of learning goal discussions per week	5
	Patient mix	Variety in patient mix	5	1.635	10.219	0.807
		Number of patients	5
		Instructional effectiveness of patient consultations	5
	Independence	Facilities to examine patients independently	5	1.294	8.087	0.719
		Opportunities to perform activities independently	5
	Information	Information about clerkship organisation	5	–	–	–

Subjects

From the total of 293 students who completed attachments in general practice in 2007, 284 had filled out the regular evaluation questionnaire (response rate 97%). One of the non-responders had dropped out of medical school but other reasons for non-response were not retrieved. Since several practices had received more than one student during the study period, some GP preceptors had been evaluated by several students. In order to obtain a sample of independent evaluations, we restricted the dataset by randomly selecting one questionnaire for each GP preceptor. The resulting sample consisted of 155 independent evaluations.

Confounders

In order to correct for potential confounders and effect modifiers we retrieved information on GPs’ age and sex, practice location (urban or rural (either > or <25,000 inhabitants, respectively)), type of practice (group or solo), affiliation to Maastricht University and GPs’ additional role as a supervisor of postgraduate trainees.

Data analysis

The responses on a 10-point scale were divided by two to obtain a uniform scale with a maximum of five for all variables. The scores on all independent variables with an interval scale were centred by subtracting the mean value in order to improve the interpretation of the regression results and avoid numerical instability in the procedure.

In order to avoid colinearity effects in subsequent analyses we reduced the number of independent variables by performing an exploratory factor analysis (Principal Components Analysis and Oblimin with Kaiser Normalisation). Factors with an eigenvalue of >1 were included for further analysis (Field 2005). Subsequent reliability analyses for each of the resulting item clusters (scales) examined internal consistency and identified items with a negative effect on reliability. By averaging the scores of the items within each scale, we obtained a reduced set of independent variables.

After the initial analyses we performed correlation analysis and multivariate regression analysis to investigate the relationships between the independent variables and their contribution to the variance of instructional quality. Confounders and effect modifiers were entered into the regression model as well in order to find a possible effect they could have on the instructional quality. We used a stepwise procedure to determine which variables should be included in the model.

The statistical software package SPSS (statistical package for the social sciences, SPSS) 16 was used for the data analysis.

Results

Students rated Instructional quality 4.15 (standard deviation (SD): 0.54) on a scale from 1 to 5. The mean age of the GP preceptors was 50 years and 80% were male. Forty-three per cent were in solo practice and 62% worked in urban areas. Nine per cent were affiliated to the university and 9% supervised postgraduate trainees as well as students.

Exploratory factor analysis

Exploratory factor analysis yielded three factors accounting for 61.20% of the total variance in the 17 questionnaire items. Reliability analysis revealed that removal of one of the items (‘information about clerkship organisation’) resulted in a considerable increase in the reliability of one of the scales (Independence), with an increase in Cronbach's alpha from 0.62 to 0.72. Based on this finding, Information was included as a separate variable in subsequent analyses. Thus, the original set of 17 independent variables was reduced to a set of four variables:

1. Supervision: quality of supervision provided by the GP preceptor,

2. Patient mix: number, diversity and quality of patient contacts,

3. Independence: opportunities and facilities to perform clinical activities independently and

4. Information: information about clerkship organisation (Table 1).

Relationships between the variables

Instructional quality correlated significantly and substantially (0.48–0.73) with Supervision, Patient Mix and Independence, but not with Information (Table 2). None of the confounders correlated significantly with Instructional quality. As for correlations between independent variables, there were significant substantial correlations between Supervision, Patient Mix and Independence (0.44–0.55), but Information showed a significant but moderate correlation with Independence only.

Table 2.  Mean scores, SDs and correlations between variables

		Mean	SDs	Supervision	Patient mix	Independence	Information
Dependent variable	Instructional quality	4.15	0.54	0.73*	0.65*	0.48*	0.09
Independent variables	Supervision	4.04	0.60	–	0.54*	0.44*	0.07
	Patient mix	4.44	0.62		–	0.50*	0.23
	Independence	4.41	0.82			–	0.25*
	Information	4.00	0.60				–

Note: *Correlation is significant at the 0.01 level (two-tailed).

Multiple regression analysis revealed that Instructional quality was directly affected by Supervision and Patient Mix, but not by Independence. Since Independence correlated significantly with all the variables, regression analysis was performed to investigate any indirect relationships with Instructional quality. We hypothesised that, if Supervision affects Patient Mix, Independence and Instructional quality and Independence affects Patient Mix, Independence could indirectly affect Instructional quality. Figure 1 shows the corresponding path diagram indicating two multiple regression Equations (1) the original equation where Supervision, Patient Mix and Independence explain the variance in Instructional quality and (2) an additional equation where Supervision and Independence explain the variance of Patient Mix.

Figure 1. Path diagram presenting the linear relations between the four variables of interest. Note: Indicated are the standardised regression coefficients (betas) that are statistically significant at p < 0.0005.

The corresponding stepwise regression analyses revealed that Supervision affects Instructional quality not only directly (regression coefficient B: 1.06; standard regression coefficient beta: 0.57) but also indirectly, with Supervision affecting Patient Mix (B 0.40; beta 0.39) and Patient Mix affecting Instructional quality (B 0.63; beta 0.37). Moreover, only for practices in urban areas, the indirect relation between Supervision and Instructional quality was found to be mediated by Independence. Supervision directly affected Independence (B 0.61; beta 0.44) and Independence indirectly affected Instructional quality through Patient Mix, with betas of 0.39 (B 0.33) and 0.37, respectively. Combined, these direct and indirect pathways between Supervision and Instructional quality result in a correlation of 0.73 between these variables. Note that there is no direct relationship between Independence and Instructional quality and in rural practices Independence did not affect Instructional quality at all, neither directly or indirectly. Rural/urban practice was the only effect modifier with a statistically significant contribution. The path diagram in Figure 1 shows the significant contributions of the independent variables of interest.

Discussion

We investigated which aspects of general practice attachments make a substantial contribution to instructional quality as perceived by students. The results show direct effects of the quality of supervision and the size and diversity of the patient mix, with supervision affecting the instructional quality the most. Moreover, supervision influences the extent to which independent activities of students are facilitated, as well as the quality of the patient mix they encounter. So, supervision has a direct as well as an indirect impact on instructional quality.

We were surprised to note that the variable independence shows a significant correlation with each variable, while having no direct impact on instructional quality. We did, however, establish an indirect relationship between independence and instructional quality via patient mix, although this is restricted to urban areas.

The importance of supervision and number and variety of patient encounters for the quality of workplace learning is supported by earlier work in this domain. Similar studies in hospital-based attachments (Wimmers et al. 2006) also show that instructional effectiveness depends on a combination of patient mix and supervision (Dolmans et al. 2002a, b). This association is confirmed by other studies reporting a positive correlation between students’ overall satisfaction on the one hand and supervision, feedback and instructional quality on the other hand (Remmen et al. 2000). Studies also show that medical students value constructive feedback from clinical staff (van der Hem-Stokroos et al. 2003), adequate numbers and variety of patient encounters and enthusiastic and available preceptors (Schultz et al. 2004).

Based on recent literature, we expected independence to be a key factor for instructional quality. As stated by Dolmans et al., input variables such as available rooms and organisational quality have a favourable effect on students’ perceptions of patient mix and supervision, while a lack of opportunities for students to examine patients independently is perceived as obstructing learning (Dolmans et al. 2002a, 2008). Moreover, preceptors who provide opportunities for students to take independent responsibility (Riesenberg et al. 2001) and be actively involved in clinical work (van der Hem-Stokroos et al. 2003) are identified as important determinants of educational value. Therefore, we were rather puzzled by the absence of a direct relationship between independence and instructional quality in the current study. A possible explanation may be found in the strong relationships between supervision and independence and between supervision and instructional quality. These relationships may leave little space for independence to come to the surface as a variable with a distinct linear relationship with instructional quality. Translated to the realities of clinical training, one could say that the quality of workplace learning in general practice is first and foremost coloured by quite intensive one-to-one supervision. The success of extensive student–teacher interaction in establishing a nourishing instructional climate determines not only general instructional effectiveness but also – and closely related to it – the extent to which students are facilitated to work independently. This strong relationship makes it difficult to tease out independence as a separate determinant of instructional quality. Another potential explanation is that the questionnaire is not yet validated and independence comprises no more than two items as opposed to the ten items of supervision. However, the differences in reliability between these variables do not account for the substantial differences in their correlations with instructional quality.

Moreover, we wondered what could cause the influence of independence on patient mix in general practices in urban areas. It is plausible that a student with many opportunities to practise independently encounters a larger and more diverse patient mix. However, neither the data nor the empiricism offers a satisfactory explanation for the fact that this relationship only occurs in urban areas. Could there be another confounder or effect modifier at work? We have to date no data that can rule this either in or out.

Strengths of this study are the large sample size and the anonymity of the questionnaire, which reduces the likelihood of socially desirable answers. As noted earlier, analysis shows that the average score of instructional quality is quite high with a moderate SD (=0.54). The relatively small variation in student ratings of the different items hampers reliable assessment of any relationships between the variables. Given the large sample size and the exploratory factor analysis we performed we are confident that this issue has been addressed adequately. By pooling the questionnaire items, we have improved the validity of our study. An inevitable downside of clustering, however, is the concomitant loss of specific information from individual items and the related reduced specificity of applications to practice.

The questionnaire being anonymous, students’ age and gender were not available for analysis. It would have been interesting to compare the effects of various combinations of student and supervisor gender, as these can play a role in teaching and learning in the student-preceptor dyad (Carney et al. 2000).

An important conclusion that we can draw from our findings is that there are few or no differences between hospital and general practice settings with regard to the key ingredients that determine students’ judgement of instructional quality. In both settings, sufficient numbers and diversity of patient encounters and high quality supervision are considered indispensable for effective learning and teaching. Opportunities and facilities for students to undertake independent activities appear to play an important role as well. Although supervision, patient mix and independence clearly are the main constituents in both hospital and primary care settings, we cannot rule out that the concrete meanings of these elements may differ between these settings. Because of differences in the organisation of health care in hospital and primary care and more specifically in view of the unexpected effects we found for independence, we will first have to explore students’ and teachers’ interpretations of these concepts before we can make pronouncements on concrete implications for educational practice with any confidence. Thus, more research is needed to verify and, more importantly, to give further meaning to our findings.

Acknowledgements

We thank C. P. M. Aretz for research assistance and L. C. Schilder for helping us out with the evaluation questionnaires. Furthermore, we thank M. Gorsira for her contribution to English grammar and style.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.