Effect of teaching context and tutor workshop on tutorial skills

Abstract

Effective faculty development workshops are essential to develop and sustain the quality of faculty's teaching. In an integrated problem-based curriculum, tutors expressed the needs to further develop their skills in facilitating students’ content learning and small-group functioning. Based on the authors’ prior observations that tutors’ performance depends on their teaching context, a workshop was designed not only tailored to the tutors’ needs but also organized within their respective teaching unit. The purposes of this study are (1) to evaluate whether this workshop is effective and improves tutors’ teaching skills, and (2) to assess whether workshop effectiveness depends on tutors’ performance before the workshop and on their teaching unit environment. Workshop effectiveness was assessed using (a) tutors’ perception of workshop usefulness and of their improvement in tutorial skills, and (b) students’ ratings of tutor performance before and after the workshop. In addition, an analysis of variance model was designed to analyse how tutors’ performance before the workshop and their teaching unit influence workshop effectiveness. Tutors judged the workshop as helpful in providing them with new teaching strategies and reported having improved their tutorial skills. Workshop attendance enhanced students’ ratings of tutors’ knowledge of problem content and ability to guide their learning. This improvement was also long-lasting. The workshop effect on tutor performance was relative: it varied across teaching units and was higher for tutors with low scores before the workshop. A workshop tailored to tutors’ needs and adapted to their teaching unit improves their tutorial skills. Its effectiveness is, however, influenced by tutors’ level of performance before the workshop and by the environment of their teaching unit. Thus, to be efficient, the design of a workshop should consider not only individual tutors’ needs, but also the background of their teaching units, with special attention to their internal organization and tutor group functioning.

Introduction

To be effective, a problem-based-learning (PBL) tutor should know well both the content of the problems and how to facilitate the small-group learning process (Dolmans et al., 2002). This has implications for recruiting tutors, in particular those who are familiar with the problem content (Wilkerson, 1994). It also highlights the need for faculty development programmes. Most programmes emphasize a general understanding of PBL and its values, exercising the tutorial process, and developing general and content-specific tutor knowledge and skills (Holmes & Kaufman, 1994; Irby, 1996). The few studies that have documented the effectiveness of these training programmes show that they help develop new tutorial skills (Wilkerson & Irby, 1998; Skeff et al., 1998; Skeff et al., 1999; Griffith, 2000; Hewson et al., 2001). They specifically indicate that workshops, followed by tutorial practice and/or feedback from students or peers, are effective strategies in improving teachers’ knowledge, attitude and performance ratings.

Table

Practice points

• Effective faculty development workshops are essential to develop and sustain the quality of teaching.

• A faculty development workshop tailored to tutors’ needs within their teaching unit improves their problem-content knowledge and ability to guide students’ learning, especially for tutors with low students’ ratings.

• To be efficient, the design of workshops should consider specificities of teaching contexts since they affect the effectiveness of workshops.

With the introduction of a new integrated six-year problem-based curriculum in 1995 (Vu et al., 1997), the University of Geneva Faculty of Medicine (UGFM) implemented a multilevel tutor-training programme. All tutors in the medical sciences years were required to attend an introductory workshop, which consisted of (1) an introduction to the curriculum, the rationale for PBL, the tutorial process and the tutor role; (2) an observation and/or participation in a tutorial session in their assigned teaching unit (to put the tutor in the teaching context); and (3) a discussion with other tutors to share observations and strategies in tutoring. Students’ ratings indicated that the workshop has adequately prepared tutors (Baroffio et al., 1999). However, although tutors improved their overall performance during their first three years of practice (Baroffio et al., 1999), we observed that students’ ratings of tutors’ abilities to give feedback and to help students synthesize the problem remained low. To assist tutors in maintaining and improving their teaching skills, we offered to all experienced tutors an advanced optional workshop, emphasizing and working on tutorial situations that they commonly identified as difficult. This second workshop improved significantly the ratings of tutors with low performance ratings, specifically their overall performance and ability to guide students’ process of problem synthesis (Baroffio et al., 1999). In addition, we showed that this workshop improved the tutors’ ratings above and beyond the self-improvement demonstrated by most tutors with increased experience in tutoring.

However, over the years we have noticed that tutor ratings tended to vary and differ according to the teaching unit they taught in. Based on our observations and on the findings that faculty development programmes are most effective when they target the learners’ needs (Davis et al., 1995; Skeff et al., 1997) and address these needs within their teaching context (Skeff et al., 1997), we designed a third advanced workshop, organized by teaching unit and focusing on the tutors’ and the unit's needs. It was introduced in 1999–2000 for all practising tutors. As the tutors’ major needs were to become better prepared in the content in order to guide students’ learning and to improve their skills in facilitating small-group functioning, the workshops were organized into two parts. The first was devoted particularly to providing the tutors with both pedagogical content knowledge and an organizational approach to their functioning within the unit by stimulating them to have regular meetings to preview cases, to negotiate key concepts to be covered in the tutorials, and to construct a common understanding of problem-content and the learning process (Holmes & Kaufman, 1994; Irby, 1996). The second part addressed small-group facilitation skills, specifically how to handle students’ or small groups’ dysfunctioning (Kaufman & Holmes, 1996), by analysing and working through various difficult tutorial situations. The content of each workshop was specifically adapted to the questions and needs of the tutors within the teaching unit, thus not only addressing their individual needs but also considering the functioning of each tutor within his teaching context (Skeff et al., 1997).

The purposes of this study are to evaluate whether this workshop is effective and further improves tutors’ teaching skills, and to assess whether workshop effectiveness may depend on the performance of tutors before the workshop and on their teaching context.

Methods

Context and subjects

This study was conducted over three academic years (1998–99; 1999–2000; and 2000–01). The second and third years of our pre-clerkship curriculum are composed of 13 teaching units (e.g. circulation, nutrition and digestion, infectious disease, etc.). Each unit is taught once a year, lasts four weeks, and consists of 16 two-hour tutorial sessions. Each tutor teaches one unit to a group of students (mean: seven students, range: four to 15). Students rate their tutor at the end of each teaching unit. Each class counts about 150 students. During the 1999–2000 academic year, 203 tutors were invited to attend an advanced workshop and were sent one month beforehand a pre-workshop questionnaire. Of these 203 tutors, 149 (73%) completed the pre-workshop questionnaire and 145 (71%) attended the workshop. Of the 145 workshop participants, 112 (75%) completed the workshop evaluation questionnaire and 78 (54%) returned the self-assessment questionnaire, which was administered one year after the workshop. Of the 203 tutors, 126 were completely evaluated by students, namely before, one month after and one year after the workshop. Among them, 96 attended the workshop and 30 could not attend for different reasons, thus constituting a convenience control group.

Needs assessment: pre-workshop questionnaire

The pre-workshop questionnaire consisted of 38 true/false items probing different activities that the tutors carried out before (15 items), during (13 items) and after (10 items) their teaching in the unit. These items mainly referred to activities defined and referred to in the literature as useful in developing and guiding students’ acquisition of problem-content knowledge and in facilitating the PBL tutorial process and small-group functioning (Holmes & Kaufman, 1994; Tipping et al., 1995; Kaufman & Holmes, 1996). In addition, tutors were asked with open-ended questions to identify aspects for which they needed specific training and teaching situations they found difficult to manage.

Intervention workshop

The design of our advanced PBL workshop for practising tutors is based on evidence-based workshop practices (Wilkerson & Irby, 1998; Griffith, 2000). Using a ‘tailored’ approach it integrates in its programme learner-centred, interactive and reflective teaching strategies (Hewson, 2000). Overall, we designed each workshop to address specifically the needs of the tutors within each of the 13 teaching units and as identified in the pre-workshop survey. The needs commonly identified by tutors consisted of how better to guide students in learning the unit problem content and facilitate small-group functioning. Towards these purposes, we planned each workshop with the teaching unit director(s). It usually took place during the month preceding the teaching period, in a session where tutors met to finalize their teaching plans. It consisted of two parts and lasted three hours. Specifically, the first part of the workshop session was set up (1) to facilitate discussions among unit directors and tutors on their perceived difficulties and (2) to help tutors reflect and discuss strategies on how better to prepare for the problem content of the teaching unit, guide students’ learning and define the depth to which students should address the learning objectives of each problem. In a second part, a situation-based approach was used to assist tutors in developing advanced strategies to facilitate small-group functioning. For this purpose, we presented pre-selected scenarios representing common difficult tutorial situations (i.e. a disruptive or silent student, a non-responsive group, …), and asked tutors to analyse, discuss and share strategies that they would use to deal with these situations. When needed, we proposed additional alternative solutions.

Instruments and outcomes measures

1. Tutors’ evaluation of the workshop. At the end of the workshop we asked the participants to rate on a five-point Likert scale questionnaire whether the workshop (a) was useful for their role as a tutor, (b) furthered their understanding of the tutorial group functioning and of the importance of small-group facilitation in PBL, and (c) provided new strategies in preparing the problem content of the teaching unit and facilitating small-group functioning.

2. Tutors’ self-assessment of changes in teaching strategies. One year after the workshop, we sent a questionnaire to all participants. They indicated on nine yes/no items changes they had adopted as a tutor and changes that occurred to the tutors’ functioning in their teaching unit. They also commented freely on the new teaching strategies they had adopted.

3. Students’ ratings of tutor performance. Each group of students rated their tutor at the end of the teaching unit. The 16-item evaluation questionnaire has been adapted from the one developed and validated at the Maastricht medical school (Dolmans et al., 1994) and has been described in an earlier paper (Baroffio et al., 1999). For this study, we selected the four items of the questionnaire that were directly related to the content of the workshop. Students were asked to rate on a five-point Likert scale (from 5 = completely agree to 1 = completely disagree) the following three items: (i) My tutor has good knowledge of the problem content of the teaching unit; (ii) My tutor guides me through the PBL process; and (iii) Students participate actively in the small group. The fourth item asked the students to rate on a three-point scale (5 = outstanding, 3 = good or 1 = to be improved) the tutor's overall performance. Students’ ratings were obtained before the workshop (tutor baseline score), then one month and one year after the workshop.

Statistical analyses

A quasi-experimental design was used to compare the 96 tutors who attended the workshop with the 30 who could not attend. The data consisted of a total of 7938 ratings performed by 450 students from the classes of 1998 to 2000. We compared students’ baseline ratings of the 126 tutors with the ratings they received one month and one year after the workshop. We conducted multivariate analyses of variance (ANOVAs) with the score differences after the workshop (one month and one year respectively) as dependent variables; workshop attendance and teaching unit as between-subject factors; baseline score and teaching experience (number of teaching years at the onset of the study) as covariates (SPSS version 11.0, Chicago). We considered an alpha value under 0.05 to be significant.

Results

Tutors’ baseline performance

Students’ ratings of tutor performance obtained before the workshop revealed marked differences not only between specific tutorial skills but also across teaching units. Students rated 12% of the 126 tutors (all teaching units combined) as insufficient in their overall performance, 13% in their knowledge of the problem content of the teaching unit, 10% in their guiding of students through the PBL process, and 38% in their ability to facilitate students’ participation to their tutorial group. Furthermore, these insufficient ratings varied across teaching units for tutors’ overall performance (0% to 36%), knowledge of the problem content (0% to 43%), PBL guide (0% to 50%), and facilitation of students’ participation (0% to 86%).

Tutors’ needs

A total of 149 tutors completed the pre-workshop questionnaire and 116 listed their specific needs. Overall, 12% of the tutors did not feel well prepared in guiding students’ learning of the unit problem content and 23% in facilitating small-group functioning. However, when each individual teaching unit was taken into consideration, a different picture emerged from the tutors’ answers and comments. There were considerable differences in tutors’ perception of their level of preparedness across the teaching units. While in some teaching units none of the tutors reported any difficulties, in some others up to 27% of the tutors felt insufficiently prepared in guiding students’ learning of the problem content, and up to 42% had difficulties in handling student and group functioning. Most importantly, in some teaching units these difficulties were raised by more than 90% of the tutors in their written comments. Many tutors also indicated a need to further define, with the tutors in their teaching unit, the level of depth to which the problem-learning objectives should be achieved.

Tutors’ evaluation of the workshop

In all, 112 workshop participants completed the workshop evaluation questionnaire (Table 1). Overall, the tutors rated the workshop as very useful for their role as a tutor. The workshop was judged helpful in furthering tutors’ understanding of the tutorial group learning process and the importance of small-group facilitation skills. It also provided them with useful strategies in improving their skills to facilitate small-group functioning and in preparing the unit problem content by working with members of the unit teaching group.

Table 1.  Tutors’ evaluation of the workshop usefulness (n = 112 tutors) on a five-point Likert scale (5 = I totally agree to 1 = I totally disagree)

Overall, the workshop:	Mean ± SD
• was very useful for my role as a tutor	4.36 ± 0.77
• furthered my understanding of the tutorial group functioning process	4.19 ± 0.82
• furthered my understanding of the importance of small-group facilitation in PBL	4.13 ± 0.82
• provided useful strategies to improve content knowledge with the group of tutors in my teaching unit	4.28 ± 0.95
• provided useful strategies to improve the functioning of small groups	4.27 ± 0.89

Tutors’ self-assessment of changes in their tutoring strategies

The self-assessment questionnaire that was administered one year after the workshop was returned by 145 workshop participants. Regarding their own performance, 58% of the tutors self-reported as more active guides of their students’ learning and 53% as better small-group facilitators, in particular by stimulating students to discuss their group functioning. Regarding their teaching unit's functioning, 43% of the tutors judged that the workshop helped modify their preparation on the unit problem content. Written comments revealed more frequent tutor meetings within the unit, active attempts at improving the quality of the problems, and in seeking consensus among tutors on the depth to which the learning objectives should be addressed and studied. However, these percentages varied across teaching units, suggesting that not all tutors benefited to the same extent from the strategies proposed during the workshop.

Change in tutor performance after the workshop

Table 2 describes students’ mean ratings of tutors’ ‘overall performance’, ‘problem-content knowledge’, ‘skill in guiding students through the PBL process’, and ‘ability to facilitate student participation’ before the workshop, one month after and one year after the workshop. Before the workshop, although tutors who attended the workshop had a slightly lower baseline score for these skills, both groups (with and without workshop attendance) did not differ significantly. The differences in scores over time between baseline and one-month and one-year scores respectively were analysed by multivariate ANOVAs. The single and interactive effects of workshop attendance, tutors’ baseline score, teaching unit and teaching experience on the score difference one year after the workshop are summarized in Table 3. This revealed that (a) the score difference in tutors’ ‘overall performance’ depended on the baseline score and on the teaching unit (61% of variance explained); (b) the score difference in tutors’ ‘problem-content knowledge’ depended on the baseline score, teaching unit, and on an interaction effect between workshop attendance by teaching unit (74% of variance explained); (c) the score difference in the tutors’ ‘skill in guiding students through the PBL process’ depended on workshop attendance, baseline score and teaching unit (55% of variance explained); the score difference on tutors’ ‘ability to facilitate student participation’ depended on baseline score, teaching unit and teaching experience (74% of variance explained). Compared with the results obtained one year after the workshop, we found no effect of the workshop or of the teaching experience on the change in tutors’ ratings one month after the workshop (results not shown). The teaching unit only affected the change in ‘student participation’. The baseline score, however, had a significant effect on all four variables.

Table 2.  Scores before workshop (baseline), one month and one year after the workshop of students’ mean (±SD) ratings of 126 tutors’ on several tutorial skills (overall performance, problem-content knowledge, skill in guiding students through the PBL process, and ability to facilitate student participation)

	Overall performance^a	Content knowledge^b	Guide in PBL^b	Student participation^b
Workshop not attended (n = 96)
Baseline	4.3 ± 0.7	4.7 ± 0.4	4.6 ± 0.3	4.1 ± 0.6
One month	4.1 ± 0.8	4.6 ± 0.4	4.4 ± 0.4	4.1 ± 0.7
One year	4.2 ± 0.9	4.6 ± 0.6	4.3 ± 0.5	4.3 ± 0.4
Workshop attended (n = 30)
Baseline	4.1 ± 0.9	4.5 ± 0.7	4.5 ± 0.4	4.1 ± 0.7
One month	4.1 ± 0.9	4.5 ± 0.7	4.5 ± 0.6	4.2 ± 0.5
One year	4.3 ± 0.7	4.6 ± 0.4	4.6 ± 0.4	4.4 ± 0.4

Notes: ^aThe ‘overall performance’ is rated on a three-point scale (5 = outstanding, 3 = good, 1 = to be improved). ^bThe ‘problem-content knowledge’, ‘skill in guiding students through the PBL process’, and ‘ability to facilitate student participation’ are rated on a five-point Likert scale (from 5 = completely agree to 1 = completely disagree).

Table 3.  Effect of workshop attendance, tutors’ baseline score, teaching unit and teaching experience, and their interactions on the score differences in students’ ratings of several tutorial skills one year after the workshop (overall performance, problem-content knowledge, skill in guiding students through the PBL process, and ability to facilitate student participation)

	Mean squares^a	F (df)^a	p^a
Overall performance:^b
workshop	0.143	0.314 (1;100)	0.576
baseline score	28.065	61.977 (1;100)	0.000*
teaching unit	1.375	3.036 (12;100)	0.001*
workshop × teaching unit	0.351	0.774 (10;100)	0.653
teaching experience	0.208	0.460 (1;100)	0.499
Content knowledge:^c
workshop	0.106	0.797 (1;100)	0.374
baseline score	18.126	135.663 (1;100)	0.000*
teaching unit	0.698	5.227 (12;100)	0.001*
workshop × teaching unit	0.357	2.672 (10;100)	0.006*
teaching experience	0.011	0.083 (1;100)	0.774
PBL guide:^d
workshop	1.071	7.776 (1;100)	0.006*
baseline score	7.619	55.308 (1;100)	0.000*
teaching unit	0.307	2.231 (12;100)	0.015*
workshop × teaching unit	0.143	1.040 (10;100)	0.416
teaching experience	0.003	0.024 (1;100)	0.878
Student participation:^e
workshop	0.085	0.610 (1;100)	0.436
baseline score	29.044	208.259 (1;100)	0.000*
teaching unit	0.299	2.141 (12;100)	0.020*
workshop × teaching unit	0.102	0.732 (1;100)	0.692
teaching experience	0.594	4.263 (1;100)	0.042*

Notes: ^aMultivariate analyses of variance with workshop attendance and teaching unit as between-subject factors; score before workshop and teaching experience as covariates. All factors and interactions were tested. Interactions are designated by ×.

^bR² = 0.612; ^cR² = 0.743; ^dR² = 0.546; ^eR² = 0.738.

In order to show separately the different effects of workshop, teaching unit and baseline score, Figure 1 displays the effects of workshop attendance and teaching unit (1a), and of workshop attendance and baseline score (1b) on the score differences after the workshop for ‘problem-content knowledge’. It illustrates first (1a) that score differences were larger for workshop participants (0.18 ± 0.65 vs −0.10 ± 0.57). Furthermore, for some teaching units, score differences after the workshop were small and did not differ according to workshop attendance (e.g. units 1, 7, 13), whereas for others they were larger for tutors who attended the workshop (e.g. units 5, 9). In addition, for the tutors who attended the workshop, the score differences after the workshop were negatively correlated to their baseline scores (1b). In other words, the lower the baseline score of a tutor, the higher was his improvement in ‘content knowledge’ after the workshop. This was also true for their ‘overall performance’ and ‘PBL guide’ (Table 4). Conversely, for the tutors who did not attend the workshop, the score differences were either weakly (for ‘overall performance’ and ‘PBL guide’) or not (for ‘content knowledge’) correlated to their baseline scores (Table 4). In other words, tutors with a low baseline score before the workshop improved more after attending the workshop compared with those who did not. On the other hand, the correlations on ‘student participation’ were similar for both groups of tutors, showing no specific effect of the workshop.

Figure 1. Effect of workshop attendance, teaching unit and baseline score on the score differences in content knowledge of 126 tutors one year after the workshop. 1a: score differences for the 30 tutors who did not attend the workshop (•) and for the 96 tutors who attended the workshop (○) for each of the 13 teaching units studied. 1b: correlation between the score difference 1 year after workshop and the baseline score obtained before the workshop for individual tutors.

No workshop (•, dashed line): Pearson's R = −0.147: p = 0.438; n = 30. Workshop (○, full line): Pearson's R = −0.802: p = 0.000; n = 96.

Table 4.  Correlations between the score difference after workshop and the baseline score obtained before the workshop for individual tutors

	Workshop not attended		Workshop attended
	Pearson's R	p	Pearson's R	p
Overall performance	−0.411	0.024*	−0.707	0.000*
PBL guide	−0.448	0.013*	−0.663	0.000*
Content knowledge	−0.147	0.438	−0.802	0.000*
Student participation	−0.715	0.000*	−0.821	0.000*

In summary, the workshop improved tutors’ knowledge of the problem content of the teaching unit and their ability to guide the learning of their students. This effect worked on a long-term basis and was more effective for low-scoring tutors but it seemed to vary across teaching units. On the other hand, our workshop did not seem to improve tutors’ ability to facilitate students’ participation, a skill that appeared, however, to improve with teaching experience.

Discussion

Our study demonstrates that our workshop, designed to address tutors’ specific needs and organized within their respective teaching unit, does help them develop new teaching strategies and improves their teaching skills. This confirms earlier findings on the effectiveness of specifically designed advanced training workshops (Holmes & Kaufman, 1994; Irby, 1996; Wilkerson & Irby, 1998; Skeff et al., 1998; Skeff et al., 1999; Baroffio et al., 1999; Hewson, 2000; Griffith, 2000; Hewson et al., 2001; Pandachuk et al., 2004). In addition, our study reveals that the effectiveness of our workshop is relative and depends on tutors’ performance before the workshop, and on the teaching unit they are teaching in.

The design of our advanced PBL workshop for practising tutors is based on evidence-based workshop practices (Wilkerson & Irby, 1998). Our strategy was to tailor the workshop to the needs of the tutors in the context of their teaching unit (Davis et al., 1995; Skeff et al., 1997). This was justified by our previous observation that tutors’ performance varies across teaching units and by this study, showing that tutors’ needs are different for each teaching unit.

Tutors reported that the workshop, as designed, provided them with a better understanding of the tutorial group functioning and the small-group facilitation processes. Moreover, they stated that they had acquired new strategies on how to develop advanced problem-content knowledge and small-group facilitation skills. They confirmed these perceptions one year after the workshop, by reporting being better guides of their students’ learning and more efficient small-group facilitators. Moreover, many tutors spontaneously recounted in free comments that the workshop modified the functioning of their teaching unit, in particular through more frequent tutor meetings. Thus, the workshop also seemed to improve the organizational vitality of the teaching units. Planning the workshop with the teaching unit director(s) and integrating it as a part of the unit regular preparation meeting for tutors presented several advantages. First, as the workshop was not perceived as an additional activity to be attended, most tutors participated, thus preventing the usual self-selection of the best and most motivated tutors who usually attend such faculty development activities (Baroffio et al., 1999; Griffith, 2000). Second, it helped create or re-create the working dynamics among the unit tutors, thus favouring the necessary discussions within their own teaching unit. Finally, it facilitated the renewing of tutorial skills immediately before tutors started teaching.

Analyses of the change in students’ ratings of tutor performance after the workshop revealed that the workshop specifically improved tutors’ problem-content knowledge and ability to guide the learning of their students. This effect was not detectable when tutors taught right after the workshop but one year later. One possible explanation for these findings could be the following: after the workshop, the group of tutors from the teaching unit meets more frequently, as reported by the tutors; this leads to an amelioration of the quality of the problems, a common understanding among tutors regarding problem content, and better concordance on the depth to which the learning objectives should be covered; as a result, the tutors get then better knowledge of the content of the problems and become better guides of the learning process. However, we found that, depending on the teaching unit they were teaching in, tutors’ problem knowledge progressed diversely after the workshop. This means that, even tailored to tutors’ needs within their teaching unit, our workshop did not seem to work for all of them. These differences among teaching units could be related to the quality of the problems, the structure of the unit, students’ prior knowledge (Schmidt, 1994), or tutorial groups’ productivity (Dolmans et al., 1999), which have been shown to influence tutor performance. More work is clearly needed to understand why and how the workshop did not elicit all the needed changes in the functioning of all teaching units.

On the other hand, we have found that the workshop predominantly improves the tutorial skills of tutors with low baseline scores, thus reinforcing our previous results (Baroffio et al., 1999). This finding might be due to the sharing of strategies between expert and/or skilled tutors with inexperienced and/or untrained tutors during the workshop and the more frequent tutor meetings of the teaching unit. This implies that even though the low-baseline tutors do benefit more from the workshops, training should not be offered to these tutors only, since the above results suggest that they learn from better tutors.

Finally, students’ ratings revealed that tutors improved their ability in facilitating student participation, and tutors assessed themselves as being better facilitators of small-group functioning after the workshop. This effect, which is also stronger for low-baseline tutors irrespective of workshop attendance, appears not to be due to the workshop but simply to teaching experience.

This study has several limitations. First, we defined our control group as those tutors who could not attend the workshop. Whereas we have no indication of the reasons why these tutors could not attend, we think that they did not self-select (“I do not need to go”), since their scores before the workshop were similar to those of the tutors who attended the workshop. They might nevertheless be influenced by secondary effects of the workshop, such as more frequent teaching unit meetings, which may limit the specific appraisal of workshop effectiveness. Second, the questionnaires that we used to assess tutors’ needs, to evaluate the workshop and to appraise tutors’ changes in teaching after the workshop have been elaborated for the purpose of this study but have not been validated at other institutions. This might represent a shortcoming of our study.

In conclusion, based on prior observations that tutors’ performance greatly depends on their teaching context, we designed a workshop to address tutors’ needs within their teaching unit. We found that this workshop improved tutors’ understanding of the PBL principles, their repertoire of teaching strategies, their knowledge of problem content and their ability to guide students’ learning. This workshop was found to be more effective for tutors who had low scores before the workshop. Furthermore, its long-term effect on tutor performance may presumably result from modifications of teaching units’ functioning. Nevertheless, despite the fact that each workshop was tailored to each teaching unit, it was not equally effective for all teaching units. We do not know the reason for this finding, but hypothesize that for many different reasons modifications in functioning may not occur in all teaching units.

Our results have implications for the design of tutor workshops, suggesting that such interventions should be tailored not only to individual tutors’ needs but also to the needs of specific teaching units, with special attention to their internal organization.

However, more research is needed to further explore the type of internal organization that improves the functioning of teaching units and hence the performance of their tutors, and to understand how a workshop can elicit a change in this functioning.

Additional information

Notes on contributors

Anne Baroffio

ANNE BAROFFIO, PhD, is assistant professor at the Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Switzerland.

Mathieu R. Nendaz

MATHIEU N NENDAZ, MD MHPE, is Médecin Adjoint of the Division of General Internal Medicine, University Hospitals, Geneva, and a faculty member of the Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Switzerland.

Arnaud Perrier

ARNAUD PERRIER, MD, is Professor of Internal Medicine of the Division of General Internal Medicine, University Hospitals, Geneva, and Faculty of Medicine, University of Geneva, Switzerland.

Carine Layat

CARINE LAYAT, MS, is research assistant at the Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Switzerland.

Bernard Vermeulen

BERNARD VERMEULEN, MD, is Médecin Adjoint and the head of the Emergency Service, University Hospitals, Geneva, Switzerland.

Nu V. Vu

NU VIET VU, PhD, is full professor and Director of the Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Switzerland.