The role of observational research in improving faculty lecturing skills: A qualitative study in an Italian dental school

Abstract

Background: This pilot study is based on observational research of lecturing skills during the annual Oral Medicine course at the Milan Dentistry School.

Aims: Our goals were to explore how teachers exhibited desirable lecturing skills, to observe how their attitudes and lecturing skills affected students’ attention and thereby learning, and to provide feedback.

Method: We prepared a structured observational grid divided into four categories: explaining, questioning, visual aids, and lecturer attitude. The grid was filled in by a participant, nonactive researcher.

Results: Two main types of lecture were observed: “traditional” and “interactive”. Both of these can result in a high level of attention among students. Among the categories, only “lecturer attitude” appeared to affect student attention. In particular, the skills of “speaking aloud” and “sustaining verbal communication with vocal inflection” appeared to have the greatest impact on lecturer attitude. The data were then presented blindly to the five lecturers, who were able to identify their own lesson.

Conclusions: Our grid proved to be a valid instrument although it was very expensive. When integrated with other strategies for improving lecturing, such as student scoring, peer evaluation, and microteaching, observational research can be a cost-effective method to stimulate guided reflection and to improve the lecturing skills of faculty members.

Introduction

Lectures are still the most widely used method of teaching in health sciences education. Some studies have shown that a teacher's expertise and lecturing skills are the two main determinants of a lecture's quality (Cox & Ewan 1988). Lecturing skills can be improved using student ratings, teacher training (including microteaching), peer evaluation, and expert observation (Brown & Manogue 2001). Although studies have stressed the importance of analyzing the teacher behavior in improving lecturing skills (Hartman & Nelson 1992), few observational studies have been conducted to examine lecturing skills during the education of health professionals.

Table

Practice points

Lectures are still the most widely used method in health sciences education. The two main determinants of a lecture's quality are teacher expertise and lecturing skills. Lecturing skills can be improved by peer and/or expert observation. Lecturer's observation by a grid is very expensive, because it requires a trained expert observer, the time-consuming collection of a vas quantity of data, and lengthy analysis. Observational analysis integrated with microteaching, lectures’ students scoring and peer evaluation, can be a cost-effective method to stimulate reflection on action and to improve faculties’ lecturing skills.

We carried out an observational study to analyze the behavior of a randomly selected sample of five of the six faculty members who teach the annual Oral Medicine course at Milan Dental School, which involved approximately 230 third-year students. The observed lectures took place between October and December 2001 (one semester) and were part of a heavy course load. The study was qualitative and based on an ethnographic approach, so the observer was a participant, but was not active.

This pilot study was part of a wider research plan, based on randomized controlled trials of web-based learning, which was incorporated into the Oral Medicine course at the Milan Dental School. Trials began during the first semester of the 2001–2002 academic year, whose material and methods have been discussed elsewhere (Canegallo et al. 2003). Such controlled trial required that we attempt to minimize other variables that could influence the learning achievements of students. One of these variables was the quality of lectures. Because the various topics within the Oral Medicine course were all presented by qualified faculty members (although they had different lengths of teaching experience), we decided to evaluate the quality of lectures only by observing the performed lecturing skills. For this reason, we planned an observational study to assess how various lecturing skills affected students’ learning.

Starting from this goal of monitoring the behavior of faculty members during lectures, we first decided which lecturing skills to observe and then established to what degree faculty members exhibited these skills. Based on a literature review, we prepared a structured observational grid to assess lecturing skills. Faculty members were informed of the trial, but did not know that they would be involved in an observational study; they were informed of this only after the data was gathered and analyzed. The lecturers were then presented with feedback about their observed lecturing skills, which provided them with an important opportunity to reflect on their lecturing activities and a way to learn from practice (Schön 1983). This experience taught us that observational research can play an important role for lecturers, allowing them to enhance their reflective practice and consequently improve their lecturing skills.

Aims of the study

Our goals were

• to investigate how the teachers of the Oral Medicine course exhibited the lecturing skills indicated as desirable by the literature;

• to analyze how the various characteristics and lecturing skills of teachers affected student attention, that is related to learning achievements;

• to investigate any relationship between teachers’ score and students attention;

• to provide participating teachers with feedback to allow them to improve their reflective practice about lecturing.

Literature review

Flanders (1966) and Hughes (1959) published seminal studies on teaching observation. Although enlightening, these studies were mostly finalized to observe teacher behavior in undergraduate education and in a small classroom context. To find other studies relevant to our investigation, we searched PubMed for “lecturing skills” or “teaching skills” in addition to “observational study/research.” We found no observational tool suitable for the aims and setting of our study, thus we decided to design a new one.

Examining the first set of references, we learned that microteaching is a method of analyzing lecturing skills during a short simulated lecture and decided to include “microteaching” in our database search. The literature revealed at least four ways to evaluate lecturing skills among faculty members: student ratings or opinions (gathered using questionnaires or focus groups), microteaching, peer evaluation, and expert observation. Direct teacher observation appeared to be the ideal way to study teaching behavior (Hartman & Nelson 1992). Teacher observation by peers or experts can provide insights that student opinions cannot provide (Brown & Manogue 2001), but it can be cost-prohibitive for observing many teachers over a long period.

Irby et al. (1976) used a literature review to identify the behaviors that produce a “good” lecture: defining learner needs, beginning with the lecture's objectives, using examples, summarizing, using handouts and visual aids to reinforce the spoken word, avoiding stereotypes, and asking for and giving feedback. They then created a lecture observation schedule in which 17 variables were divided into two categories: observation of instruction and observation of personal attributes. Items in the first category included the teacher's ability to communicate the lecture's objectives, use examples, use a pace appropriate for note taking, prepare appropriate supplementary resources, cite authorities to support statements, stimulate questions from students, and summarize. Items in the second category included the teacher's ability to demonstrate positive interpersonal relationships with students, exhibit confidence when teaching, and use a well-modulated speaking voice.

Peer evaluation studies, mainly conducted in the field of clinical education, provide further indications on teaching skills (Irby 1983). However most of the categories included in the tool employed in such studies (Stanford Faculty Development Program-26, SFDP-26) are suitable for clinical and small group teaching only.

In addition to studies based on expert/peer observation, studies based on microteaching produced interesting findings about what to observe in a lecture, and consequently, how to improve lecturing skills. Microteaching is a method based on videotaping a brief simulated lecture; the video is then discussed by the lecturer, the group, and one or more experts (van Ort et al. 1991). The main limitation of microteaching is the fact that it is based on a simulation, which may or may not reflect reality (Hartman & Nelson 1992). In any case, microteaching requires the individuation of some lecturing skills, including: organization of presentation; use of objectives; clarity of concepts and pacing of materials; use of questions, examples, and discussion; use of audiovisual aids; and use of summaries (van Ort et al. 1991). Irby et al. (1976) also identified most of these skills.

Another aspect of a lecture that can be observed is how it is structured by a teacher. Brown and Manogue (2001) identified different methods of structuring lectures: classical, from signs and symptoms to therapy; problem centered; sequential, in which a case is presented and followed by a chain of reasoning that leads to a solution; comparative, in which two or more perspectives are compared; and thesis, in which an assertion is made and then supported or disproved. They pointed out that these methods can be more or less difficult to follow based on a student's learning preference and his or her familiarity with a certain topic. However, they listed some basic lecturing skills that are independent of the methods of structuring lectures: opening, explaining, presenting information, narrating, comparing, and contrasting, designing and using audiovisual aids, responsiveness to the audience, varying student activity, and summarizing. Brown and Manogue (2001) claimed that explaining is a key skill for teaching and includes the essential features of clarity, generating interest, and persuasion. Clarity can be improved to minimize vagueness and sharpen focus by using structuring aids (i.e., using signposts, indicating frames, highlighting key points, showing links) and to provide a clear lecture structure. Teachers can improve student interest by using examples and a narrative mode of explaining and to stimulate curiosity. Finally, a credible expert lecturer can persuade an audience by presenting it with appealing arguments.

Another key skill in teaching is questioning, which can apply to both the questioning of students and the ability to stimulate questions from students. It is a fundamental strategy to make lectures interactive; “interactive lectures promote active involvement and increased motivation and attention by the learner, which lead to a 'higher level’ of learning, improved feedback to teacher and students, and increased teacher and student satisfaction” (Nasmith & Steinert 2001).

The design and use of visual aids is also an essential lecturing skill. Visual aids can increase clarity and interest and thereby improve understanding. Still, “the effectiveness of a particular medium depends not so much upon the medium per se, but how it is used” (Brown & Manogue 2001), so it is important to observe how the teacher uses visual aids.

Finally, an effective lecture requires good verbal and nonverbal communication. The lecturer must demonstrate a good level of expressiveness: eye contact, body movement, facial expression, vocal inflection, and choice of words. Conversely, a teacher must be able to read the expressiveness of students by being open, listening, and continuously reading their faces. Immediate reactions are usually nonverbal signals, and a lecturer can receive and interpret these signals and possibly act upon them (Brown & Manogue 2001).

Overall, it is important to note that lecturing is not only a skill, but also a strategy and overall, a practice (Young & Diekelmann 2002). Therefore, learning to lecture effectively requires not only training, but also a reflective practice (Schön 1983). As stated for Peer Observation of Teaching (POT), lecturing observation must be implemented “in such a way that it can truly foster a culture of personal questioning, reflection, adaptation, and improvement” (Siddiqui et al. 2007). These findings will be useful during our discussion of using observational research to improve lecturing skills.

Subjects and methods

Sample of observed lecturers

Data were collected during a series of lessons from the annual third-year Oral Medicine course at the University of Milan's Dental School. All students were required to take this course. The researcher responsible for data collection attended eight consecutive lessons, without knowing which teacher would be lecturing or what the topic would be. The teachers had never met the researcher and were unaware of her presence in the classroom; this prevented the Pygmalion effect, i.e., enhanced performance due to expectations of the researcher. During the eight lessons, the researcher observed five of the six teachers involved in the course. Table 1 summarizes the characteristics of the five observed teachers and the lesson topics.

Table 1.  Characteristics of the observed lecturers

	Age	Gender	Teaching experience (years)	Academic position	Lecture no.	Topic	Observation length (min)
Lecturer A	51	M	20	Full professor	1	The first visit in the oral medicine clinic: Signs and symptoms and dentist–patient relationship	75
Lecturer B	30	F	3	Clinical tutor	3	Ulcers of the oral mucosa	95
					4	Viral conditions of the mouth	79
Lecturer C	29	F	2	Clinical tutor	2	Lumps of the mouth	91
					7	Pigmented lesions of the oral mucosa	62
Lecturer D	34	M	5	Assistant researcher	5	Bollous diseases	93
					8	White lesions of the mouth	88
Lecturer E	43	F	1	Clinical tutor	6	Inflammatory, autoimmune, and neoplastic conditions of the salivary glands	87

Data collection method

We prepared an observational grid based on the lecturing skills identified as fundamental in previous studies (Irby et al. 1976; van Ort et al. 1991; Brown & Manogue 2001). The researcher involved in the data collection (S.Visioli) tested the grid during the first lesson of the course, which was presented by the teacher with the most teaching experience. We made minor adjustments to the grid based on this test to develop the final version of the structured grid (Table 2).

Table 2.  Observational grid for lectures

Topic^1:
I. Explaining	Frequency	Quality^2	Timing^3
a. Introduces the topic (today we will talk about … now we are introducing …)
b. Explains/justifies (a new term, an affirmation)
c. Clarifies (a topic, the reasoning …)
d. Makes examples
e. Synthesizes
II. Questioning
f. Asks for questions
g. Picks up spontaneous interventions from class
III. Visual aids
h. Writes on the blackboard
i. Uses slides
j. Uses images, pictures
k. Other (specify)
IV. Lecturer attitude	When^4	Quality^5
l. Reveals a dialogic attitude
m. Uses nontechnical language
n. Speaks aloud
o. Sustains verbal communication with vocal inflection
p. Catches expressiveness of class
q. Sustains verbal communication with nonverbal communication (eye contact, facial expression, body movement)

Note: ¹One grid for each topic.

²Insert, for each behavior observed, the perceived quality: 1 = fair, 2 = sufficient, 3 = good, 4 = excellent.

³Register each behavior length.

⁴Specify if in category 1 (explaining), 2 (questioning), or 3 (use of visual aids).

⁵Insert, for each behavior observed, the perceived quality: 1 = fair, 2 = sufficient, 3 = good, 4 = excellent.

The grid was divided into four categories, each comprising different items: I. Explaining (items a–e); II. Questioning (items f–g); III. Use of visual aids (items h–k); IV. Lecturer attitude (items l–q). The frequency of use, quality, and length (in seconds) were recorded for each item from categories I, II, and III. Items related to lecturer attitude (Category IV) were observed in relationship to skills from categories I, II, and III. Lecturer attitude was therefore observed globally by indicating specific usage: during explaining (Category I), questioning (Category II), or when using visual aids (Category III). For instance, when a teacher was explaining, the researcher recorded whether he/she had a dialogic attitude (item l), used nontechnical language (item m), etc. In addition, the researcher scored how she perceived the quality of each item (a–q) on a scale of 1–4. For instance, item d or f received score 4 when, respectively, the teacher used specific examples/questions about a topic.

Every lesson was audiotaped. To avoid the bias due to the so-called “Pygmalion Effect”, we decided not to use a videorecorder, and we preferred to audiotape, a very common practice among Italian students (the observer being one among them).

During lessons, the researcher filled in the grid for Categories III (Use of visual aids) and IV (Lecturer attitude), whereas data about Categories I (Explaining) and II (Questioning) were filled in later as the researcher listened to the recording. The same researcher repeated (and checked) the data collection by listening to the audiotape a second time.

We chose not to videotape lectures because the study did not focus on observing specific nonverbal communication skills (for instance, “scans the audience”). The researcher started a new grid for each topic discussed by the teacher during the lesson, so that one or more grids were filled out for each lesson.

At the end of the lesson, the researcher produced a short global assessment (based on her perception of quality) and recorded student behavior during the lesson on the basis of noise, note taking, and feedback (globally scored as “level of student attention/interest”). For example, the researcher noted whether the students were making noise or interfering with the teacher speaking. She graded student behavior using the following scale: 1 = low attention; 2 = moderate attention; 3 = good attention; 4 = excellent attention. Learning achievements were not measured at the end of lectures.

All observational grids were filled out by the same researcher (S. Visioli), who is an experienced pedagogist in the field of observational studies within an educational setting. She had no specific knowledge of dentistry. Observation of lecturing skills requires a specific expertise in pedagogy and training in observational methods, but not a specific knowledge about the subject of the lecture observed. The researcher sat in the classroom among the students; she was not recognizable and did not actively participate in the lesson (i.e., by asking questions). The researcher timed the duration of each lesson to measure what proportion of the lesson was dedicated to the activities included in the grid. Because our grid did not include all lecturing skills (i.e., narrating) we expected the total length of each lesson (lesson time) to differ from the combined length of the various recorded lecturing skills (grid time).

Data analysis methods

Data collected were analyzed using Microsoft Excel 2000. For every lesson, we obtained an aggregate result for each item by pooling all data related to the item regardless of the topic, e.g., “explains” was combined with “justifies.” Because we obtained data for three variables (i.e., frequency, quality, and length) for each item (a–k), each lesson resulted in a total of 33 values. We calculated the frequency of each item, compared these across the eight lessons, and analyzed their relationship with the level of student attention.

Results

The observed lecturers exhibited familiarity with the lecturing skills previously identified as desirable. All lecturers used the lecturing skills included in our grid with varying frequency; the total time devoted to grid categories I and II also varied, ranging from 10% to 35% of the total lesson time. When examining this data, it is important to keep in mind that our grid was not intended to measure all teaching behaviors (i.e., “making a list” or “commenting on pictures”), but only those which are considered key behaviors for a good lesson.

Tables 3 and 4 summarize the frequency of items a–k during the eight lessons, divided into three categories: Explaining (Table 3), Questioning (Table 3), and Use of visual aids (Table 4). These tables also indicate the scores for student attention levels. Table 5 presents the scores assigned by the expert observer to the quality of lecturing skills included in Category I (Explaining), II (Questioning) and III (Visual aids).

Table 3.  Frequency of items included in category I (explaining) and category II (questioning)

	Lesson
	1	2	3	4	5	6	7	8
CAT I: Explaining
a. Introduces	18	10	16	19	12	15	14	16
b. Explains	21	14	25	22	21	22	7	10
c. Clarifies	26	20	15	9	5	6	9	10
d. Makes examples	27	8	22	9	9	5	2	6
e. Synthesizes	22	16	13	22	19	9	8	27
Total CAT I	114	68	91	81	66	57	40	69
CAT II: Questioning
f. Asks for questions	6	22	4	8	4	15	28	31
g. Picks up intervention	0	2	11	5	2	16	1	29
Total CAT II	6	24	15	13	6	31	29	60
Level of student attention	4	1	3	4	2	3	2	4

Data related to items l–q from Category IV (Lecturer Attitude) represent teacher attitude during the activities described by the other three categories (Explaining, Questioning, and Use of visual aids). We pooled all data from items l–q related to these three activities and calculated the results for each lesson (Table 6). Table 6 also includes the scores for student attention levels.

Table 4.  Frequency of items included in Category III (use of visual aids)

	Lesson
	1	2	3	4	5	6	7	8
CAT III: Visual aids
h. Blackboard	0	0	0	0	0	0	0	0
i. Slides	31	2	11	14	56	10	4	6
j. Images	19	14	37	22	29	39	10	16
k. Other	0	0	0	3	0	0	0	0
Total CAT 3	50	16	48	39	85	49	14	22
Level of student attention	4	1	3	4	2	3	2	4

We examined teacher characteristics and the use of lecturing skills to investigate whether these factors significantly affected student attention levels during the lesson (Tables 1–4). There was no significant relationship between the level of student attention and single teacher characteristic such as age, gender, or academic position (Table 1).

Table 5.  Scores assigned by the expert observer to the quality of lecturing skills included in categories I, II, and III

	Lesson
	1	2	3	4	5	6	7	8
Quality CAT I Explaining	4	3	4	3	3	3	3	3
Quality CAT II Questioning	4	3	3	4	4	4	3	4
Quality CAT III Visual aids	3	4	3	3	3	3	4	4
Mean quality	4	3	3	3	3	3	3	4
Level of student attention	4	1	3	4	2	3	2	4

The frequencies and distributions of single lecturing skills varied greatly among lessons; no single pattern appeared to capture maximum attention. For instance, Lecture 1 had a high frequency of lecturing skills related to Category I (Explaining) and resulted in a high level of student attention. Lecture 8 had a high frequency of lecturing skills related to Category II (Questioning) and also resulted in a high level of student attention. If a lecture that contains high levels of Category I (Explaining) skills is labeled “traditional” and a lecture containing high levels of Category II (Questioning) skills is labeled “interactive,” then both traditional and interactive lectures appear to be able to achieve similar levels of student attention (Figure 1).

Figure 1. Frequencies of category I and II lecturing skills as observed during the lessons.

For the lecturing skills included in Category I (Explaining), there was a negative relation between low frequency of the skill “introduces the topic” (item a) with class attention (Lecture 2). In contrast, there was no significant correlation between frequency of “makes examples” (item b) and student interest (Lectures 1 and 8), nor was there any significant correlation between the skill “clarify the terms” (item c) and student attention (Lectures 2 and 4).

Table 6.  Frequency of items included in category IV (lecturer attitude)

	Lessons
	1	2	3	4	5	6	7	8
CAT IV: Lecturer attitude
l. Dialogic attitude	3.4	3.1	3.2	3.7	3.6	3.2	3.5	4.0
m. Language	4.0	3.0	3.9	3.7	3.5	3.5	3.1	4.0
n. Speaks aloud	4.0	2.5	3.9	4.0	3.8	3.6	3.3	4.0
o. Vocal inflection	4.0	2.3	3.6	3.9	3.4	3.2	3.4	3.7
p. Catches expressiveness	4.0	2.5	3.9	4.0	4.0	4.0	3.6	3.7
q. Nonverbal communication	3.6	2.3	3.2	3.4	3.8	4.0	3.5	3.9
Mean	3.8	2.6	3.6	3.8	3.7	3.7	3.4	4.0
Level of student attention	4	1	3	4	2	3	2	4

For the lecturing skills included in Category II (Questioning), during Lecture 8, a high frequency of these skills was associated with the highest level of student attention. However, during Lecture 1, the skill “asks for questions” appeared with low frequency and still resulted in good attention levels. Interestingly, the frequency of this item (item f) appeared to be negatively correlated with the frequency of skills from Category I (Explaining (Lectures 7 and 8).

Lectures that received the highest level of attention tended to use a very traditional or very interactive teaching approach. Interestingly, Lecture 2 received the least attention and used lecturing skills from Categories I and II (Explaining and Questioning) with a frequency equivalent to the averages for Category I and II skills over all lectures. It is possible that students prefer either very traditional or very interactive lectures, rather than a combination of the two. These data suggest that even if teachers use lecturing skills from Categories I and II (Explaining and Questioning), this may not result in a high level of student attention.

We also examined the use of visual aids. The data suggest that the more frequently a lecturer used the skill “asks for questions” (item f), the less he/she used visual aids (Lectures 7 and 8). In addition, neither the quantity nor the quality of visual aids predicted student attention (Tables 4 and 5). Lecture 5 had many visual aids, but this did not result in the maximum level of student attention. Lecture 2 had a lower-than-average number of visual aids (16 versus the average of 40) and achieved low classroom attention.

The only variable that appeared to predict student interest was lecturer attitude (Table 6). Lecturer attitude was scored highest during Lectures 1 and 8, which received the maximum student attention. The skills of “speaking aloud” (item n) and “sustains verbal communication with vocal inflection” (item o) appeared to have the greatest impact on lecturer attitude. In addition, lecturer attitude seemed to be most relevant in interactive lectures.

We presented all of the collected data blind to the five lecturers. Each lecturer was able to identify his/her lesson from the descriptions made by the two educationalists (L. Zannini and S. Visioli). This feedback allowed lecturers to explore their lecturing skills, compare them to those of their colleagues, and discuss the various class reactions to the different lecturing skills. During this reflective practice, we noticed that the lecturer with the longest teaching experience had also had the best education in medical teaching and used most of the teaching skills with the highest awareness.

Discussion

All of the observed lessons received a score of medium to high quality from the expert observer. These data were not always consistent with student attention (Table 5), which might have been affected by the large number of students in the classroom (230 students) and the heavy course load. The lecturers who effectively used skills from Category I (Explaining) achieved high student attention, but there was no direct correlation between the frequent use of lecturing skills from Category I and student interest. In fact, lessons with the greatest student attention did not always have the highest frequency of skills from Category I (e.g., Lectures 2 and 8). The use of skills from Category I can pique student interest and thus learning achievements, but other factors are likely involved.

Our analysis of specific skills from Category I indicated that student interest was increased by introducing the topic (item a) so that students understood what the lecture would be about. Furthermore, student attention appeared to be maintained when the lecturer synthesized the lecture step-by-step (item e).

The use of the lecturing skills from Category II (Questioning) should result in an attention-grabbing lesson, but the data indicate that, as with Category I, there was no direct relationship between frequent use of skills from Category II and student attention (e.g., Lectures 1 and 2). As with results for Category I, other factors probably contribute to these results.

Both Lessons 1 and 8 achieved high levels of student attention: Lesson 1 had frequent use of skills from Category I, which is typical for a traditional lesson; Lesson 8 had frequent use of skills from Category II, which is typical for an interactive lesson. Lesson 2 fell somewhere between a traditional and an interactive lesson and produced low levels of student interest. This suggests that clearly traditional or clearly interactive lecturer behavior will best maintain student attention.

All of the lessons that had the highest scores for student attention also had high quality scores for questioning skills (Table 5), but these lessons did not have the highest frequency of questioning skills (Table 3). Furthermore, there was no direct relationship between quality scores for lecturing skills from Category I (Table 5) and student interest or between quality scores for visual aids and student interest (Table 4).

The most traditional lesson (Lesson 1) and the most interactive lesson (Lesson 8) produced the highest levels of student interest. Nevertheless, our data does not allow us to infer that an attention-grabbing lesson can be created using only a high frequency of skills from Category I or Category II.

In contrast, our data indicate that there is some relationship between the level of lecturer attitude and student attention: a high level of student interest was observed in all lessons (traditional or interactive) that received high scores for lecturer attitude (Table 6). Our findings indicate that none of the lecturing skills that are typically considered desirable can be predictive of student interest alone. Student attention is best piqued and maintained by appropriate vocal inflection, expressiveness, use of words that are not excessively technical, and clarifying the topic through nonverbal communication (i.e., indicating the location of pain). Therefore, a lecturer should always incorporate desirable lecturing skills with a good attitude.

A dialogical attitude (i.e., asking rhetorical questions) is important to a good lesson. However, we found that asking the students general questions (i.e., “Any questions?”) did not improve their interest in the lesson. The real difference between a “good” and a “bad” lesson appears to be the ability to activate students using specific questions about a topic, rather than simply questioning. In this way, the lecturer helps students to reflect upon the topics of the lecture.

The most interesting result is the capacity of our grid to represent teaching behavior. Although we presented the results blind to lecturers, i.e., they did not know which data were from which lecturer, almost all teachers could recognize data related to their own lessons. This mirroring prompted discussion and reflection among lecturers, allowing them to become more conscious of their teaching behavior and to compare it with that of their colleagues. This reflection is the main goal in the use of observational grids to assess a sample of lecturers: observational research should not be implemented only to assess class attention, but mainly to help lecturers to reflect on their teaching practice.

Limits of the study

We analyzed the influence of certain lecturing skills on student interest during lessons taught by a random sample of teachers within a specific course (Oral Medicine). This random sample may limit our study, but it included 5 out of 6 who taught this course. Another limitation is the fact that we observed only lecturers for a specific course. Because this research is qualitative, the results about the relationship between lecturing skills among teachers of Oral Medicine and student attention are not necessarily applicable to other situations.

The data were collected by only one observer; nevertheless, observation using a structured grid reduced the degree of observer bias. The structured grid is a useful instrument to observe the context of a lesson, in which many events occur simultaneously. Videotaping could have been a method to overcome the limit of the single observer, as it allows to observe the lectures more than once and by more than one observer. However, in our study teachers and students did not know when they would have been observed and by whom; we know that presence of technological instruments can seriously affect people's attitude and behavior, this being the reason why we choose to employ audiorecorder. In future observational studies it might be useful, although not cost-effective, to have two researchers in the class, who, following calibration, observe the lessons and assess the quality of the different items of the grid (column “Quality” in Table 2).

Another limitation is that we did not evaluate student learning at the end of each lesson to compare learning achievements with the type of lecturing skill and student attention. However, the evaluation of student learning is difficult because it depends on the student's previous knowledge and other learning opportunities, in addition to lecturer expertise and lecture quality. For these reasons, observational research about lecturing skills should primarily focus on the reflective practice of lecturers, rather than the identification of the most effective lecturing skills or the improvement of student learning.

Conclusions

Observational research can contribute to the improvement of lecturing skills because it can help faculty members to reflect on their own lecturing practice. Furthermore, by comparing their lessons with those of colleagues, teachers had the opportunity to reflect on different ways of lecturing. Although our grid was demonstrated to be a valid instrument, observational research about lecturing skills is best done among a small number of teachers. In fact, as suggested by previous studies, expert use of a grid is very cost-prohibitive, requiring a trained observer, time-consuming collection of a vast quantity of data from various observational grids, and lengthy data analysis.

Many possible strategies could be used to improve lecturing skills, e.g., student scoring, microteaching, peer evaluation, and observational research. These strategies would be best used in an integrated manner; the use of observational research alone requires too many resources. A combination of strategies would improve lecturing skills in a cost-effective manner. Under any conditions, guided reflection based on data collected using an observational grid could be an important strategy for stimulating discussion among lecturers and for improving learning from practice. In conclusion, the analysis and discussion of data collected through observational research, ideally integrated with student scoring, peer evaluation, and microteaching, can effectively stimulate guided reflection on lecturing and improve the lecturing skills of faculty members.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Lucia Zannini

LUCIA ZANNINI, SONIA VISIOLI and ANTONIO CARRASSI were responsible for the study concept and design. Literature search was carried out by LUCIA ZANNINI. Data analysis was done by GIOVANNI LODI and SONIA VISIOLI. GIOVANNI LODI, LUCIA ZANNINI and SONIA VISIOLI prepared the manuscript, which was reviewed by all the authors.