10,217
Views
13
CrossRef citations to date
0
Altmetric
Research Article

Quality of teaching in higher education: reviewing teaching behaviour through classroom observations

ORCID Icon, &
Pages 31-44 | Received 08 Nov 2019, Accepted 07 Jun 2020, Published online: 15 Oct 2020

ABSTRACT

This study examines the extent to which lecturers demonstrate effective teaching behaviour. The results of 203 observations reveal substantial differences in detected teaching behaviour. Lecturers mostly demonstrated teaching behaviour in the domains classroom climate, efficient organisation, and instruction. Teaching behaviour relating to the domains activating teaching and teaching learning strategies was observed less frequently, with almost no evidence of behaviour associated with the domain differentiation. The quality of teaching in small classes was slightly higher than that in large classes. These findings can help tailor professional learning activities to lecturers’ professional development needs.

Introduction

Effective professional learning activities focus on meeting the professional development needs of individual lecturers (Jacob et al., Citation2015). To balance various needs, educational developers mainly support lecturers in two central practices: course planning and classroom teaching (Hora & Ferrare, Citation2013). While considerable literature has been published on course planning practices, relatively few studies have focussed on characterizing lecturers’ classroom instruction. This is surprising since research has clearly shown the quality of teaching behaviour is associated with student achievement (Hattie, Citation2015; Schneider & Preckel, Citation2017). Furthermore, the descriptions of teaching practices in the literature rely mostly on lecturers’ self-reports, while actual teaching behaviour is hardly captured (Stes et al., Citation2010). To fill this gap, our study aims to identify to what extent lecturers demonstrate effective teaching behaviour. Data will be collected through classroom observations. Whereas other researchers have examined the frequency of teaching behaviour in the higher education classroom (Hora & Ferrare, Citation2013; Stains et al., Citation2018), this study focuses on the quality of teaching behaviour.

Theoretical framework

Effective teaching behaviour

The literature has long focussed on dichotomising teaching practices in either teacher-focused or student-focused behaviour (Postareff et al., Citation2008; Trigwell & Prosser, Citation2004), where the latter is considered to be more effective. However, teaching is complex and various teaching behaviours affect student achievement (Hattie, Citation2015; Schneider & Preckel, Citation2017). The authors have recognised this and have recently moved beyond the dichotomisation of teaching by categorising teaching behaviour in various domains (Hora & Ferrare, Citation2013; Stains et al., Citation2018). The categorisation still simplifies teaching to a certain extent, but helps educational developers to support lecturers in developing their teaching behaviour by focusing feedback on specific teaching domains (Maulana et al., Citation2015). From the literature we derived six observable teaching behaviour domains that are known to impact student achievement: (1) safe and stimulating learning climate, (2) efficient organisation (classroom management), (3) clarity of instruction, (4) intensive and activating teaching, (5) teaching learning strategies, and (6) differentiation.

A safe classroom learning climate has been defined as a climate ‘that allows students to feel secure enough to take risks, honestly express their views, and share and explore their knowledge, attitudes, and behaviours’ (Holley & Steiner, Citation2005, p. 50). Teaching behaviour that promotes such a climate has been found to have a medium-large effect on student achievement (Schneider & Preckel, Citation2017) and might be especially important in classrooms with a diverse student population (Crose, Citation2011).

Classroom management is needed for dealing with apparent incivilities, such as talking in class and dominating a discussion (Burke et al., Citation2014; Nilson, Citation2010), close supervision and monitoring of students’ performances, and using classroom time efficiently (Biggs & Tang, Citation2011; Kim et al., Citation2014). Other scholars have debated the relevance of classroom management in higher education on the grounds that students at this level have been in formal education for longer than secondary school students and the student population is more homogenous (William T. Grant Foundation, Spencer Foundation & Bill & Melinda Gates Foundation, Citation2014). However, in Feldman’s (Citation2007) meta-analysis, classroom management is mentioned as a dimension of moderate importance for student achievement (r = 0.26).

Many studies have included clear and structured instructions as an essential criterion of quality teaching in higher education (e.g. Feldman, Citation2007; Hativa et al., Citation2001). Schneider and Preckel (Citation2017) identified that clear and understandable instructions had the strongest association with student achievement in their category ‘presentation’ (d = 1.35).

There is compelling evidence that confirms the impact of activating teaching on student outcomes. The meta-analysis of Freeman et al. (Citation2014) found that active learning increased students’ grades by almost half a standard deviation (0.47) in STEM undergraduate education. Schneider and Preckel’s (Citation2017) review study of meta-analyses also highlighted the importance of engaging and interactive teaching, both in lectures and small group settings.

Learning strategies are broadly defined as ‘cognitions or behaviours that influence the encoding process and facilitate acquisition and retrieval of new knowledge’ (Fryer & Vermunt, Citation2018, p. 22). As noted by Kirschner et al. (Citation2006, p. 77), ‘the goal of instruction … is to give learners specific guidance about how to cognitively manipulate information in ways that are consistent with a learning goal.’ This applies particularly to the context of higher education, which is expected to generate strategic, self-regulated, lifelong learners who are able to analyse new information, apply their knowledge to real-world problems, and reflect on proposed strategies (Biggs & Tang, Citation2011; Ramsden, Citation2003). Vermetten et al. (Citation1999) provided a longitudinal perspective on the development of learning strategies within higher education. Their results suggest that guidance on the development of student learning strategies impacts students’ application of concrete deep-level learning strategies.

Insights derived from studies conducted at lower educational levels have prompted some scholars to argue in favour of a more differentiated instructional approach within higher education to address an increasingly diverse student population (Chamberlin & Powers, Citation2010; Ernst & Ernst, Citation2005; Santangelo & Tomlinson, Citation2009). The few studies examining the impact of differentiated instruction in higher education report promising results, such as higher student satisfaction (Ernst & Ernst, Citation2005) and a positive impact on student learning (Chamberlin & Powers, Citation2010; Santangelo & Tomlinson, Citation2009).

These domains can be classified as foundational or advanced. As described above, without a safe and stimulating classroom climate, efficient organisation of the lesson, and clear instructions, it will be difficult to implement successful activating teaching. Furthermore, studies that focused on the teaching behaviour of student teachers (Van de Grift et al., Citation2014) and teachers in secondary education (Van der Lans et al., Citation2018) observed, on average, behaviour in the first three domains (foundational skills), while the latter domains were found to be more challenging (advanced skills). These findings are in line with teaching development theories that describe the advancement of teaching as starting with a focus on oneself as a teacher, focusing on the task, and finally focusing on student outcomes (Akerlind, Citation2003; Fuller, Citation1969).

Aim and hypotheses

The present study aims to contribute to the literature by observing to which degree lecturers demonstrate effective teaching behaviour. Following earlier studies in secondary education (Van de Grift et al., Citation2014; Van der Lans et al., Citation2018), we hypothesize that lecturers demonstrate more foundational than advanced teaching behaviour. Our second hypothesis is that the number of students in a class impacts teaching behaviour; lecturers teaching small classes demonstrate more advanced teaching behaviour than lecturers teaching large classes. Large classes bring unique challenges to the teaching and learning environment, such as classroom management (Biggs & Tang, Citation2011; Mueller & Schroeder, Citation2018), developing students’ academic skills (Clarence et al., Citation2014), and implementing differentiated instruction (Turner et al., Citation2017).

Methods

Sample

The sample comprises 203 observations. A total of 211 university lecturers were observed once. Of these observations, six were discounted as they entailed student presentations with little lecturer involvement. One observation was discounted because the observer withdrew from the study, and one observation was deleted because the observation took place in an atypical room (a room in the local cinema). The male–female ratio in the sample is 119:84. Most lecturers (73%) taught a course in the soft sciences compared to 27% in the hard sciences. More bachelor (75%) than master courses (25%) were observed. Courses were typically scheduled for seven weeks. Teaching experience ranged from lecturers who just started teaching to lecturers who had been teaching for 44 years (M = 11.1, SD = 9.1). The class sizes varied between 5 and 220 students (M = 44.9, SD = 45.4).

Observers and observation procedure

Following ethics approval, we sent out invitation letters to all lecturers (over 400) who we knew would be teaching in the second semester (February to June 2017). We did not distinguish between academic rank or type of contract (full-time versus sessional staff). In most faculties, lecturers were asked to voluntarily subscribe to the sample pool. In one faculty, they were informed of the observations and given the option to opt out of the pool.

A total of 25 observers were recruited for the observations. Fourteen educational advisors who mainly focus on professional learning of lecturers were asked to participate as observers. In addition, 11 master students studying in the field of Social Sciences were recruited as observers. Two days of training were organized (one for the educational advisors and one for the students) to increase consensus among the observers and to ensure inter-rater reliability. The training comprised three parts: (a) an explanation and discussion on how to use the instrument and on how to carry out the three-staged observation process consisting of a pre-observation conversation, the actual observation, and a post-observation conversation, (b) observation and debate centring, respectively, on two recorded lessons from a course in the soft sciences and one course in the hard sciences, and (c) logistical information and ethical guidelines.

To match observers with lecturers we created a register in which lecturers’ names were replaced by unique numbers. It contained the locations and dates of the observed teaching moments. After observers selected their observations based on the information in the register, they received the lecturer’s contact details. Each observer was asked to perform ten observations and to decline those of lecturers whom they knew personally.

Measure

Observation instrument

Authors across secondary and higher education cluster teaching behaviour in similar domains. This similarity in clustering is noticeable in the various observation domains (see Appendix A). In our search for an observation instrument we excluded domains that refer to non-observable teaching behaviour, such as planning and preparation in Danielson’s (Citation2013) Framework for Teaching (FTT), and technology in the Teaching Dimensions Observation Protocol (TDOP; Hora et al., Citation2013).

We opted to use the International Comparative Analysis of Learning and Teaching (ICALT) observation instrument for this study. The ICALT clusters observable teaching behaviour in six teaching domains and includes a three-item student engagement scale, which is used as a criterion variable. Observers rate all items on a four-point scale, ranging from 1 (mostly weak) to 4 (mostly strong). Although the ICALT has been primarily used in secondary education, we believe the instrument to be suitable for observations in higher education. First, the instrument covers all the concepts applied in the other instruments and comprises items that are based on educational effectiveness research (Hattie, Citation2008). Second, the instrument reflects a cumulative ordering of teaching skills, corresponding to Fuller’s (Citation1969) stage theory of teacher development (Van der Lans et al., Citation2018). This stage-wise development has also been found in lecturers’ teaching conceptions (McLean & Bullard, Citation2000). Third, the ICALT instrument has been found to be reliable in various settings (Maulana et al., Citation2016; Van de Grift et al., Citation2017), demonstrating significant predictive validity between teachers’ demonstrated behaviour and students’ engagement.

Following Van de Grift et al. (Citation2014), who tailored the original ICALT instrument to the secondary education context (32 items), we adapted this instrument for the higher education context (see Appendix B). First, we tailored its language and formulation. For example, ‘The lesson builds up logically going from the simple to the complex’ was rephrased as ‘The lesson builds up logically (e.g., going from the simple to the complex, comparing different cases etc.)’. Second, we formulated three additional items on the distinctiveness of teaching in higher education based on our literature review. The following two items were added to the domain clear and structured instructions: ‘The lecturer presents societal or research developments of the topic’ (item 16), and ‘The lecturer compares different theories, perspectives or methods’ (item 17). Both items were inspired by Marsh’s (Citation1982) concept of ‘breadth of coverage’, referring to how scholarship is conveyed in teaching. The link between teaching and research is a fundamental characteristic of research-oriented higher education institutions. Lecturers should be aware of current developments in their fields ‘to support students to realise not only the contested uncertain nature of knowledge, but also the importance and the fascination of pursuing that knowledge’ (Healey & Jenkins, Citation2007, p. 53). The third item, ‘The lecturer teaches learners how to reflect’ (item 35), was added to the domain teaching learning strategies. Teaching students how to reflect is a universal aim of universities. However, most learners do not spontaneously develop reflection skills. To foster the development of these skills, pedagogical interventions are required across programmes and courses (Ryan & Ryan, Citation2013, Citation2013).

Validity

We examined the construct validity of the observation instrument by calculating correlations of the six domains with total scores for teaching skills (). A lecturer’s total score for teaching skills comprised the sum of this individual’s scores for the 35 items. The correlations ranged from .42 (differentiation) to .89 (intensive and activating teaching).

Table 1. Descriptive statistics and correlations (Spearman’s rho) between domains of teaching behaviour, student engagement scale, and class size (N = 203)

The predictive validity of the instrument was considered by correlating the total score for teaching skills with the student engagement scale. There was a strong, positive correlation between the two variables (r = .61, p < .001). Furthermore, the student engagement scale correlated positively and significantly with all six domains, with values ranging from .32 (safe and stimulating learning climate) to .56 (intensive and activating teaching). After applying a Bonferroni correction, the correlations remained significant (p-values of all seven pairs < 0.001).

Reliability

We calculated Cronbach’s Alpha for each domain (). The reliability ranged from .45 (efficient organisation) to .83 (teaching learning strategies), indicating internal consistency for four out of the six domains. Removing item 6 (‘the teacher monitors to ensure learners carry out activities in the appropriate manner’) increased the Cronbach’s Alpha value for the efficient organisation scale from .45 to .63. Deleting item HE1 (‘the teachers presents societal or research developments of the topic’) increased the Cronbach’s Alpha value for the clear and structured instructions scale from .68 to .73. For this descriptive study, we decided to retain items 6 and HE1 with the proviso both here and in the discussion that the results for the respective domains should be interpreted with caution. The reliability of both the total score for teaching skills and the student engagement scale was found to be good (Cronbach’s Alpha = .90 and .85, respectively). The mean inter-scale correlation ranged between .23 (differentiation) and .51 (intensive and activating teaching) indicating there is some overlap among the six domains, but they also measure separate aspects of teaching behaviour (Maulana et al., Citation2016).

Data analysis strategy

After exploring the reliability and validity of the adapted ICALT instrument, we conducted descriptive analyses to examine to which degree lecturers demonstrate effective teaching behaviour. We calculated the mean scores of the teaching domains and used the following criteria: a mean score for a domain that was below 2.5 was considered insufficient, a mean score of 2.5 up to 3.5 was considered sufficient, and a mean score of 3.5 or higher was considered excellent. This classification corresponds to a four-point (1–4) scoring scale for items included in the instrument. When a behaviour was not demonstrated, hardly demonstrated, demonstrated, or very strongly demonstrated, the scoring was 1, 2, 3, and 4, respectively.

To consider the influence of class size on demonstrated teaching behaviour, an independent sample t-test was used to examine how the demonstrated teaching behaviour differs among lecturers who taught classes of varying size. The non-parametric Mann–Whitney U test was used to study how class size impacts the teaching domains that violated the assumption of normality. To categorise classes in small and large classes, we followed the categorisation used by Biggs and Tang (Citation2011). A small class consists of no more than 30 students (n = 116). Large classes hold more than 30 students (n = 87). The class size categories were collapsed for the initial descriptive analyses. Finally, we examined whether lecturers demonstrated a different degree of insufficient, sufficient, and excellent behaviour in small and large classes following the descriptive analyses as explained earlier. All analyses were performed in SPSS 25.0.

Results

shows the distribution of the mean score per teaching domain. Most of the lecturers in the sample (68.4%) obtained an excellent mean score in the domain safe and stimulating learning climate. None of the lecturers scored insufficient and 31.5% obtained a sufficient mean score. The majority of the lectures scored sufficient in the domains efficient organisation (66.5%) and clear and structured instructions (65%). Whereas almost none of the lecturers (1.5%) obtained an insufficient score in the domain efficient organisation, 26.1% of the lecturers scored insufficient in the domain clear and structured instructions. The reverse is true for the excellent scores in both domains: 32% of the scores in the domain efficient organisation and 8.9% of the scores in the domain clear and structured instructions. Overall, the majority of lecturers demonstrated sufficient behaviour for the instrument’s first three domains. These domains reflect foundational teaching skills (Van de Grift et al., Citation2017).

Table 2. Observed domain-wise teaching behaviour categorised as insufficient, sufficient, or excellent

Observed behaviour relating to the instrument’s last three domains (reflecting advanced teaching skills) were comparatively less observed. The scores of 52.7%, 42.4%, and 4.9% of lecturers were, respectively, insufficient, sufficient and excellent for the intensive and activating teaching domain. The results for the teaching learning strategies domain were similar: the respective scores of 58.1%, 38.5% and 3.4% of lecturers were insufficient, sufficient and excellent. Of note are the scores obtained for the differentiation domain. Compared with their scores for other domains, the majority of lecturers (95.1%) attained insufficient scores for this domain and only 4.9% sufficient ones. Finally, although the majority of the observed lecturers obtained a sufficient average total score for teaching skills (58.1%), a relatively large percentage scored insufficiently (39.4%).

To examine whether the differences between observation scores of lecturers teaching small and large classes are significant we performed an independent sample t-test (). Class size had a significant but small effect on lecturers’ observation scores in the domains clear and structured instructions (ds = 0.35), teaching learning strategies (ds = 0.41), and efficient organisation (ds = 0.48). The impact of class size was larger on the scores in the domain intensive and activating teaching (ds = 0.69) as well as on student engagement (ds = 0.53). In all four domains lecturers teaching small classes scored higher than those teaching large classes. Students in small classes were also more engaged.

Table 3. Results of the independent-sample t-test for four teaching domains and the student engagement domain by class size

A Mann–Whitney U test was performed for the two remaining domains, safe and stimulating learning climate and differentiation. Non-significant differences between lecturers were found for both domains (Uclimate = 5045.5, z = −0.00, p = 1, r = 0.00; Udifferentiation = 4615.5, z = −1.14, p = .26, r = 0.08). Both findings were anticipated because of the lack of spread of the scores within these domains.

Conclusion and discussion

In this study we provided a detailed snapshot of the quality of observed teaching behaviour in one semester at a research-oriented university. Our findings indicate lecturers demonstrated teaching behaviour that was deemed sufficient in the three domains reflecting foundational teaching skills. More advanced teaching behaviour, such as intensive and activating teaching and the teaching of learning strategies, was observed less frequently. While lecturers might have the required knowledge related to these domains, they often struggle with implementing this knowledge into practice (Sadler, Citation2012). This means targeted support is needed to translate knowledge into behaviour.

The two domains that stand out in particular are activating teaching and differentiation behaviour. It is striking that almost half of the observed lecturers did not demonstrate sufficient activating teaching behaviour since the importance of active learning for student achievement has been well-established in higher education (e.g., Freeman et al., Citation2014). When considering the implications of class size on this behaviour, our results revealed that more activating and intensive teaching behaviour was noticed in small classes. The number of students might make lecturers uncomfortable in trying out active learning methods and the learning space of large classrooms might interfere with organising, for example, group work. That being said, 42.2% of the lecturers teaching small classes demonstrated insufficient activating teaching behaviour. These results are in line with Stains et al. (Citation2018). The second result that stands out is that almost no differentiation behaviour was detected. The absence of this behaviour could be attributed to the fact that lecturers might not be familiar with this concept. Differentiation has only recently been brought to the attention of higher education institutions. Another explanation might be that a course is typically taught over a seven-week period which allows lecturers little time to get to know their students. Had the courses been longer we might have observed more teaching behaviour in the domain differentiation. It is also possible that lecturers differentiated in their teaching but that it was not noticed by the observers because they were unable to identify weaker students or elucidate how the lecturer accounted for inter-learner differences. Furthermore, differentiation also occurs beyond the classroom at the university under study; it is incorporated into course designs or at the programme level, for example, in the honours programme. The honours programme allows top students to follow an additional multidisciplinary programme next to their regular studies.

Documenting teaching behaviour with an observation instrument could support the observer and the observed lecturer by providing them with a common language regarding teaching behaviour. When providing feedback on teaching behaviour, educational developers should follow the advice of observational frameworks (e.g., Bell, Citation2001; Bell & Cooper, Citation2013) and start with a pre-observation conversation in which the goal and the context of the lesson is discussed. Although all teaching behaviour domains impact student learning, the characteristics of the lesson might be a reason for a lecturer to refrain from showing a certain behaviour. When the observation instrument is structurally implemented within an institution, the results can contribute to evaluating the progress of educational innovation projects.

Limitations

As teaching practices vary across higher education institutions, the generalisability of this study’s results might be limited due to its data collection at one university in the Netherlands. The scope of the study was further limited in terms of accessing course information and lecturers’ intentions. More work needs to be done to develop a full picture of lecturers’ teaching behaviour. We will discuss three ideas for future research. First, since preparation and planning are important aspects of teaching quality (Feldman, Citation2007; Schneider & Preckel, Citation2017), further studies could explore whether a lecturer had set out to teach in an activating way but had failed to do so. Second, additional data on the place of the observed course in the curriculum, and on whether or not attendance is compulsory, is needed to fully understand the relationship between student engagement and teaching behaviour as observers often indicated that the number of students attending classes was lower than expected. Third, although the observers in this study were trained to ensure interrater reliability, it would be interesting to examine the interrater reliability of the observations if they were to observe in pairs.

Unlike the ICALT instrument used for secondary education, the adapted instrument used in this study has not yet been psychometrically validated. Furthermore, Cronbach’s Alpha value (α = .45) was low for the efficient organisation domain. At the final meeting with the observers after all observations were completed, the suggestion to further specify examples of good teaching behaviour in the instrument was proposed. For example, ‘the lecturer explains clearly which materials can be used’ (example of good teaching behaviour for item 7 ‘provides effective classroom management’) could be improved by adding ‘the lecturer connects the topic with the self-study materials’. In addition, even though the instrument was adapted for this study, the higher education context could have influenced how observers attributed meanings to the items. For example, ‘the lecturer monitors to ensure learners carry out activities in the appropriate manner’ (item 6) might have been differently interpreted in the higher education context because of the strong emphasis on students’ responsibility for learning and the corresponding perception of the role of a university lecturer. As previously noted, deletion of item 6 increased Cronbach’s Alpha value for the scale from .45 to .63. We decided to retain this item because it became apparent during the final meeting with the observers after the observations that it could be especially important for student engagement in large classes during interactive instructions. Nevertheless, the literature suggests that the efficient organisation domain is moderately important for higher education. Therefore, further scrutiny to determine the fit between this domain and the higher education context is required. In a follow-up study we will draw on the work of Van de Grift et al. (Citation2014) to analyse the psychometric characteristics of the adapted ICALT instrument and to examine the specific stage-wise ordering of teaching behaviour in higher education.

Supplemental material

Supplementary Material 1

Download PDF (275.6 KB)

Supplementary Material 2

Download PDF (212.7 KB)

Acknowledgements

We thank all of the observed lecturers and observers for their participation in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed at https://doi.org/10.1080/1360144X.2020.1830776.

Additional information

Notes on contributors

Ine Noben

Ine Noben is an educational developer and PhD student at the University of Groningen in the Netherlands. Her research interest is in the quality of teaching and professional development in higher education.

Jan Folkert Deinum

Jan Folkert Deinum is a senior educational developer and researcher at the University of Groningen in the Netherlands. His research interest is in the professional development of teachers, academic leadership, and quality assurance in education.

W. H. Adriaan Hofman

W. H. Adriaan Hofman is a professor of education studies at the University of Groningen in the Netherlands. His research focuses on school and teacher effectiveness, higher education, research methods, and education in developing countries.

References