A Global Measure of Professional Learning Communities

ABSTRACT

The concept of professional learning communities (PLCs) has received considerable attention in research as well as in school practice since the late 1990s. PLCs have been positively associated with a variety of outcomes for both teachers and students, but differences in the way the concept is operationalised, and the fact that most of the research is related to specific national contexts, make it difficult to accumulate and compare research related to PLCs. In order to give PLC research an international scope, the aim of this paper is to develop a global measure of PLCs. The paper includes an overview of the literature on PLCs, as well as an examination of existing measures. Informed by these findings, multi-level confirmatory factor analysis is applied to develop a measure of PLCs in the Teaching and Learning International Survey 2018. This measure includes three distinct dimensions and an overall measure of PLCs at school level in 42 countries/economies, and is available for researchers interested in the concept of PLCs and international perspectives. The measure is applied in an example which investigates the relationship between PLCs and teacher job satisfaction in the Nordic countries. Strengths, limitations, international comparability and possibilities for future developments are discussed.

KEYWORDS:

• Professional learning communities (PLCs)
• Teaching and Learning International Survey (TALIS)
• International Large Scale Assessment (ILSA)
• Scale Development
• Confirmatory factor analysis (CFA)

Introduction

The concept of the professional learning community (PLC), which can be defined briefly as ‘a group of people sharing and critically interrogating their practice in an ongoing, reflective, collaborative, inclusive, learning-oriented, growth-promoting way’ (Stoll et al. 2006, p. 223), has received considerable attention internationally since it emerged in the late 1990s. While many scholars have investigated the relationship between PLCs and various aspects within schools, most of the research is based on data from specific national samples. Most of this research is in the context of the United States (Bolam et al. 2005, Lomos et al. 2011a), while there is less research in other contexts (Sleegers et al. 2013, Bellibas et al. 2016, Zhang et al. 2020), and even fewer studies that have an international scope (Vieluf et al. 2012). Additionally, there are inconsistencies in the ways in which the concept is defined and operationalised, making it difficult to compare and accumulate research.

Arguably, the field of research could benefit from a global measure of PLCs that can facilitate research with an international perspective, as well as validating existing findings in other national contexts. This can only be done by using international studies which provide substantial information about different educational systems by using the same instruments across countries. The Teaching and Learning International Survey (TALIS) is specifically relevant for this ambition since it includes a great deal of information about teachers, principals and schools in 47 countries/economies.

Within this context, the aim of this paper is: 1) to outline the concept of PLCs by providing various theoretical perspectives and describing key elements; 2) to highlight inconsistencies and similarities between theory, practice and research related to PLCs; 3) to develop a global measure of PLCs at school level in TALIS 2018, drawing upon common conceptualisations, and 4) to demonstrate the utility of this measure with an example of the relationship between PLCs and teacher job satisfaction in the Nordic countries.

PLCs in theory and practice

Defining the concept of PLCs

PLCs are essentially concerned with improving instructional quality through an ongoing, practice-orientated collaboration of the teachers that provide it. For decades, the development of teacher capacity has been seen as an important aspect of school improvement, with the ultimate aim of supporting student learning. This endeavour has led to a substantial amount of literature on teacher professional learning or teacher professional development, including a wide range of activities. A broad definition of teacher professional development is ‘ … activities that develop an individual’s skills, knowledge, expertise and other characteristics as a teacher’ (OECD 2009). It is argued that this learning or development takes place in three overlapping and recursive systems: the individual teacher, the school, and the activity in question (Opfer and Pedder 2011). In this broad area of literature, PLCs stand out by emphasising that learning is situated, and that it should be developed in the context of the teacher’s actual practice. It is also emphasised that of the development of this situated learning is a continuous process that involves collaborative work and knowledge sharing with colleagues, in the context where this knowledge is generated and has its validity (Putnam and Borko 2016, Stoll et al. 2016). The concept has similarities with earlier pedagogical concepts, such as action research, communities of practice and the learning organisation (Stenhouse 1975, Senge 1990, Lave 1991).

While the concept of PLCs has become popular during recent decades, it lacks a clear definition and is used ambiguously within research. Central aspects of the concept go back to school research in the 1960s, as described below, but ‘Professional Learning Communities: Communities of Continuous Inquiry and Improvement’ by Hord (1997) is regarded as the origin of the concept and the first systematic description of the professional learning community (Hargreaves and Fullan 2012). Hord suggests:

… professional learning communities produce positive outcomes for both staff and students. For staff, being part of a professional learning community reduces teacher isolation, increases commitment to the mission and goals of the school, creates shared responsibility for the total development of students, creates powerful learning that defines good teaching and classroom practice … (Hord 1997, p. 1).

Since then, much has been written about the concept of PLCs. So much, in fact, that Richard DuFour wrote the article ‘What Is a Professional Learning Community?’ in 2004 as an attempt to define the key elements at a time when the concept, as described by DuFour, was at risk of losing all meaning because it was being used in very different contexts and with different meanings (Dufour 2004). A recent and more comprehensive definition of PLCs is provided by Hargreaves and Fullan (2012), by breaking down the meaning of each of the three words (community, learning, and professional):

1. Communities, where educators work in continuing groups and relationships, where they are committed to and have collective responsibility for a common educational purpose, and where they are committed to improving their practice in relation to that purpose.

2. Learning communities, where improvement is driven by the commitment to improving students’ learning, well-being, and achievement, where the process of improvement is heavily informed by professional learning and inquiry into students’ learning and into effective principles of teaching and learning in general, and where any problems are addressed through organisational learning in which everyone in the organisation learns their way out of problems instead of jumping for off-the-shelf, quick-fix solutions.

3. Professional learning communities, where collaborative improvements and decisions are informed by but not dependent on scientific and statistical evidence, where they are guided by experienced collective judgement, and where they are pushed forward by grownup, challenging conversations about effective and ineffective practice (Hargreaves and Fullan 2012).

Having established the notion and context of PLCs, the next section will describe the most important elements of PLCs.

Key elements of PLCs

In the following, the main aspects of how PLCs are envisioned in practice will be described. The aim is to clarify how teachers and other staff are expected to behave in practice, and which organisational features are necessary for effective PLCs. While some of the elements described naturally overlap, it is an attempt to divide the concept of PLCs into smaller segments which in sum encompass PLCs at the school level.

Shared vision

In creating PLCs in schools, the development of a shared vision (or values or beliefs) is frequently mentioned (Albrechtsen 2013, Olivier and Huffman 2016). Shared visions are intended to guide the behaviour of individuals within an organisation, moving the school as a community closer to realising this vision. The teachers should be involved in the development of this vision, the aim of which is to reach a consensus by sharing and discussing knowledge. The idea is that the focus on this shared vision, in time, will result in changes in the attitude and behaviour of individuals, thereby changing the culture at the school. Although the staff should be involved in this development, it is essential that the overall goal of this vision is focused on the learning of the students, which of course imposes some clear boundaries (Dufour et al. 2016). This emphasis on student learning leads to the next element.

A strong focus on (individual) student learning

It is an assumption of PLCs that the core mission of schools is to ensure that every student learns, not just that the students are taught. This has implications for how teachers should view their task. It is not sufficient to make sure you have taught a subject, because what really matters is whether or not the students have actually learnt what you intended. This makes the continuous monitoring of student learning (formative evaluation/assessment) imperative, making it necessary to set goals that can be evaluated in terms of what the individual students have learnt. This raises the importance of using data in monitoring the learning of the individual student, and precludes the use of class averages in monitoring student learning (Dufour 2004). The formative evaluation of student learning is central to the concept of PLCs, and teachers must clarify exactly what the students must learn, continuously monitor the degree to which this learning takes place, and develop systematic interventions that support students who experience difficulties in learning. Practice must be evaluated not in terms of the intention behind it, but in terms of whether or not it facilitates the intended student learning. The development of practice with the aim of increasing student learning is a continuous process (Dufour et al. 2016).

Collaborating and reducing teacher isolation

What actually goes on in the classroom, which traditionally has been a private matter for the individual teacher, must become subject to discussion in a systematic process in which teachers collaboratively analyse and improve classroom practice. An important aspect of this ambition is the review of classroom practice by peers (peer supervision/observation), with the aim of improving the professional practice of the individual and the community. This requires respect, trust, and the willingness to debate, accept feedback and change (Hord 1997).

Teachers must recognise that this form of collaboration is necessary in order to enhance their professional practice, and thereby student learning. The willingness to disagree and discuss the evidence and improvements of practice, which in turn will benefit the students, is the heart and soul of a strong and sustainable PLC (Dufour 2004, Hargreaves 2007). By sharing and discussing concrete experiences and testing possible solutions and new ideas collaboratively, the learning of each student becomes a project for the group and not the individual teacher, which leads to the next key element.

Shared responsibility for student learning

When teachers collaborate and discuss their professional practice, the learning of the individual student is a collective responsibility. As described by Dufour (2004), every professional in the building must engage with colleagues in the ongoing exploration of three crucial questions:

• What do we want each student to learn?

• How will we know when each student has learned it?

• How will we respond when a student experiences difficulty in learning?

Notice the ‘we’ in the three questions, highlighting the cooperative aspects of PLCs instead of the individual teacher. DuFour highlights the third question as one that separates PLCs from traditionally organised schools: remedies and interventions for students struggling with learning are an issue for the entire community, and not just the individual teacher. The emphasis on collective responsibility for student learning is also emphasised by Hord (1997) and Dufour et al. (2016) in describing schools with successful PLCs.

Leadership and supportive conditions

The final key aspect of PLCs in practice is the need for leadership support. Without the support of the school management for the development of the community, and prioritising that the teachers continuously devote time and effort to the work done in their respective communities, the ideal of PLCs is unlikely to become a reality. As stated by Mclaughlin and Talbert (2006, p. 56):

Administrators who use their authority to build a teacher community convey new expectations for teachers’ work in the school, and they ensure that teachers have the time, space, and knowledge resources needed for collaborative work.

Stoll et al. (2006) also maintain that it is difficult to see how a PLC can develop without active support from the leadership, and that distributed leadership aligns well with the idea of PLCs because it can provide the opportunity for teachers to take the lead on the development of their practice.

Measurements of PLCs in research

The following section will present an overview of strategies applied in measuring PLCs in quantitative research, with a focus on differences in terms of which aspects are included, and how the different approaches align with the theoretical framework. As mentioned in the introduction, there is a considerable amount of research related to the concept of PLCs, and within this research there is general consensus that the concept lacks a clear definition or conceptualisation. This ambiguity in terms of definition and operationalisation makes it difficult to compare and accumulate research related to PLCs (Stoll and Louis 2007, Watson 2014, Kruse and Johnson 2016).

Although there is no clear definition, a great deal of research has defined PLCs using various combinations of the following elements: shared vision/values, a strong focus on (individual) student learning, collaborating and reducing teacher isolation, and shared responsibility for student learning and inquiry (Vescio et al. 2008, Lomos et al. 2011b, Lee and Kim 2016, Prenger et al. 2018).

Other studies use additional, fewer or different aspects than those explicitly described in the review above. For example, Hurley et al. (2018) include democratic environment, teacher efficacy and confidence and the extent to which teachers feel valued in the measurement of PLCs; while Vanblaere and Devos (2016) include reflective dialogue, and ‘The Professional Learning Community Assessment-Revised’ (PLCA-R) includes the dimensions shared and supportive leadership, supportive conditions – relationships and supportive conditions – structures (Olivier and Hipp 2010). While some researchers include job satisfaction and commitment as well as mutual trust, respect and support between colleagues in measuring PLCs (Bolam et al. 2005, Sigurðardóttir 2010, Chen et al. 2016), others view these as mediating aspects or outcomes of PLCs (Zheng et al. 2016, Doğan and Adams 2018).

One important distinction between many of the existing measures of PLCs is whether or not leadership support is included as an element of PLCs. One possible explanation for this difference is that the concept of PLCs in research has emerged from two different strains of theory. According to Zhang et al. (2020), one of these strains is rooted in Senge’s work on the learning organisation, where PLCs are regarded as school-wide phenomena including cultural norms and values. This strain is closely related to the conceptualisation presented by Hord (Hord 1997), which includes shared and supportive leadership as a prerequisite of PLCs. The second strain of theoretical research is more closely related to Wenger’s concept of communities of practice, and within this perspective PLCs are perceived as communities that consist only of teachers who meet in teams, subgroups or subject departments. This strain of research is thereby aligned with the research that defines PLCs as having five elements but does not include leadership. In sum, the learning organisation perspective perceives leadership and supportive conditions as basic characteristics of PLCs, while the community of practice perspective considers leadership and school structure to be external to the concept of PLCs, and treats these elements as crucial factors influencing PLCs (Zhang et al. 2020).

It is clear from this overview that while there are many communalities, there are also distinct differences between these ways of measuring PLCs, and we still do not have a clear, universal definition. These differences within this field of research make it very important to pay close attention to conceptualisation, possible limitations related to the theoretical descriptions, and the multidimensional nature of PLCs (Hairon et al. 2017, Doğan and Adams 2018). With these lessons in mind, the following section will present the development of global measure of PLCs based on TALIS 2018 data, which aligns with the Senge/Hord approach, in which PLCs are perceived as a school-level phenomenon in which values, norms and leadership support are included.

Developing a global measure of PLCs in TALIS 2018

The ultimate aim of this paper is to develop a measure of PLCs that can be used in the international (pooled) data of TALIS 2018, as well as in a wide range of different national contexts, where PLCs are currently understudied, and where differences in data sources and measures applied make it difficult to compare results. The TALIS study is arguably the most suitable data source for developing such a measure, as its extensive teacher and principal questionnaire is administered to nationally representative samples in 47 countries/economies. Besides using the TALIS data for analysis on country-level and in comparisons between countries, using the pooled TALIS data is a strategy applied in several recent studies, controlling for country differences to test hypotheses across this extensive international data source (Liu and Bellibas 2018, Bellibaş et al. 2021, Jerrim and Sims 2021).

Like the previous cycles, the 2018 cycle of TALIS includes a wide variety of questions related to teacher practice, collaboration and professional development, but no official measure of PLCs. Closest related to the ambition of this paper is an official OECD report, using data from the 2008 cycle to investigate teaching practice and pedagogical innovation, in which PLCs are measured using a relatively simplistic measure including six items to measure five latent constructs (Vieluf et al. 2012). The same items are also used to measure PLCs in two additional articles using TALIS 2008 data, with minor changes in the first of these (Lee and Kim 2016, Doğan and Yurtseven 2018). To the author’s knowledge, no measure of PLCs has previously been created or validated to fit the combined TALIS data, and/or using data from the latest cycle in 2018.

Methods and procedure

PLCs are envisioned as school-wide communities, which is one of the aspects that distinguishes them from communities of practice, for instance. This indicates that PLCs should be measured at school level. At the same time, the core of any PLC is the teachers collaborating at the school in question, and consequently the measure should be created using responses from the teachers within the school. Recent methodological articles in multilevel modelling suggest that a ‘shared cluster construct’ is the most suitable method in pursuing this ambition. The following sections will describe the methods and procedures, the data and the final measure of PLCs in TALIS 2018.

A shared cluster construct using multilevel confirmatory factor analysis

When data is hierarchical, e.g. teachers nested in schools, and the researcher is interested in the school level, individual-level items can be used to measure characteristics of the cluster in a shared cluster construct using multilevel confirmatory factor analysis (ML-CFA). One prerequisite for such a measure is that the responses given by individuals within the same cluster are highly correlated, which can be assessed by the intraclass correlations ICC 1 and ICC 2, measuring the degree of clustering and the reliability at the cluster level, respectively (Bliese 2000). In such a model, any variance at the individual level is not of interest. Therefore, the construct of interest is specified at the between level, using the individual-level responses. Simultaneously, a saturated model of variances is constructed at the within level (Stapleton et al. 2016, Bonito and Keyton 2019). The data was prepared using the statistical software ‘R’ and the ‘MplusAutomation’ package, and CFA analysis was performed using Mplus version 8.5 (Muthén and Muthén 2017, Hallquist and Wiley 2018, R Core Team 2020).

Since the items included in the final model are all ordered categorical, weighted least square mean and variance adjusted (WLSMV) would theoretically be the ideal estimation technique (Brown 2015). However, because of a series of limitations related to using WLSMV, robust maximum likelihood (MLR) is applied. Major limitations of using WLSMV in this case are that it is not possible to export factor scores and to do multiple regression, which significantly limits the use of the PLC measure in secondary analysis. Moreover, OECD applied MLR for the official teacher scales, and another study has shown no substantial difference between the two in TALIS 2018 data (OECD 2019, p. 209, Zakariya 2020). A series of comparisons between the two estimators have similarly shown no substantial difference in the case of the PLC measure developed in this article. The R-script for processing the data and the Mplus syntax for the ML-CFA are available in the supplemental materials, along with a SPSS-file including all original TALIS data and the PLC scales, allowing researchers to apply the measures using their preferred statistical software.

Procedure adopted

Based on the theoretical review and existing measures of PLCs in research, a very broad selection of theoretically relevant items was selected from the TALIS 2018 teacher questionnaire (OECD n.d.). This initial selection of items was reviewed by two international scholars with experience in working with TALIS data and the concept of PLCs. Based on their feedback, multiple items were dropped mainly because of theoretical arguments, limitations in using binary items and to avoid differences in answer types within dimensions. The full preliminary selection of items is presented in Appendix 1.

As this structure of PLCs has not been validated previously, it would be sensible to perform exploratory factor analysis (EFA). However, the specific way in which a shared cluster construct is operationalised does not translate directly into the EFA framework. Therefore, EFA was only used to explore how the items loaded together at the individual level, in order to inform the changes to the ML-CFA model at the early stages. The procedure involved in developing the final measure was to test different competing models aligned with theory, assessing changes in model fit and factor loadings as the basis for and restructuring and reducing dimensions and items. The final model was selected based on the best possible theoretical coverage as well as acceptable model fit statistics (Thompson and Daniel 1996, Brown 2015).

Data

TALIS 2018 includes 47 countries/economies in the publicly available data from the core survey of teachers and principals in the UNESCO International Standard Classification of Education (ISCED) level 2, which is the data used in this analysis. The sample strategy involved randomly selecting 20 teachers from 200 schools. Different stratification strategies have been applied across the participants (OECD 2019).

Results

The final model consists of 14 items that are used to measure three latent constructs: collaborative practice with focus on student learning (CPL), shared vision and responsibilities (SVR), and supportive conditions (SC). These three latent constructs are used to create the overall measure of PLCs at school level. The items that comprise the three latent constructs and the answer categories are presented in Table 1, and the final model is illustrated in Figure 1 along with the standardised factor loadings.

Figure 1. Final model with standardised factor loadings, pooled data.

Table 1. Factors and items in final model of PLCs.

Collaborative practice with focus on student learning
Main question text: On average, how often do you do the following in this school?	Answer categories Never Once a year or less 2–4 times a year 5–10 times a year 1–3 times a month Once a week or more
Items
tt3g33a	Teach jointly as a team in the same class
tt3g33b	Observe other teachers’ classes and provide feedback
tt3g33d	Exchange teaching materials with colleagues
tt3g33e	Engage in discussions about the learning development of specific students
tt3g33f	Work with other teachers in this school to ensure common standards in evaluations for assessing student progress
tt3g33h	Take part in collaborative professional learning
Shared vision and responsibilities
Main question text: How strongly do you agree or disagree with these statements, as applied to this school?	Answer categories Strongly disagree Disagree Agree Strongly agree
Items
tt3g48a	This school provides staff with opportunities to actively participate in school decisions
tt3g48d	This school has a culture of shared responsibility for school issues
tt3g48e	There is a collaborative school culture which is characterised by mutual support
tt3g48f	The school staff share a common set of beliefs about teaching and learning
Supportive conditions
Main question text: How strongly do you agree or disagree that the following present barriers to your participation in professional development?	Answer categories Strongly disagree Disagree Agree Strongly agree
Items
tt3g28c*	There is a lack of employer support
tt3g28d*	Professional development conflicts with my work schedule
tt3g28f*	There is no relevant professional development offered
tt3g28g*	There are no incentives for participating in professional development

*: The answer categories are reverse coded for the purpose of this analysis.

Table 2 shows the ICC 1, ICC2 and the amount of missing data for each item. The ICC 1 varies from 0.09 to 0.37, and the ICC 2 is within the range of 0.63 to 0.91. For the ICC 1, Lebreton and Senter (2008) suggest that a value of 0.01 is a small effect, a value of 0.10 is a medium effect, and 0.25 is a large effect. Similarly, Fleiss (1999) suggests that in assessing reliability, as measured by the ICC 2, values below 0.4 indicate poor reliability, values between 0.4 and 0.75 indicate fair to good reliability, and values above 0.75 represent excellent reliability. In sum, all the items in the final model demonstrate satisfactory levels of clustering and reliability, making them suitable for modelling a shared cluster construct.

Table 2. Item statistics for pooled data.

	Variable	ICC 1	ICC 2	Missing data
CPL	tt3g33a	0.23	0.84	2.93%
	tt3g33b	0.37	0.91	2.97%
	tt3g33d	0.18	0.79	2.86%
	tt3g33e	0.20	0.81	3.07%
	tt3g33f	0.12	0.70	3.09%
	tt3g33h	0.26	0.86	3.23%
SVR	tt3g48a	0.16	0.75	3.19%
	tt3g48d	0.14	0.73	3.56%
	tt3g48e	0.15	0.75	3.51%
	tt3g48f	0.15	0.75	3.47%
SC	tt3g28c	0.21	0.82	3.26%
	tt3g28d	0.12	0.70	3.10%
	tt3g28f	0.09	0.63	3.13%
	tt3g28g	0.16	0.77	3.04%

The ICC 2 value is calculated using the Spearman-Brown formula, using the ICC 1 value and the average cluster size (Bliese 2000).

Figure 1 illustrates the final model and includes the standardised factor loadings. All the factor loadings are statistically significant (p ≤ 0.01). For the factors SVR and SC, the loadings range from 0.679 to 0.949. In the CPL factor, the items tt3g33a and tt3g33b have relatively low factor loadings of 0.282 and 0.355, but have been kept in the model owing to the theoretical importance, the fact that the loading is statistically significant, and the fact that the model fit is good. The three latent constructs CPL, SVR and SC all demonstrate statistically significant loadings on the overall PLC construct at school level in the range from 0.494 to 0.744.

A range of fit statistics aligned with common recommendations for reporting CFA models are presented in Table 3. Hu and Bentler (1999) recommend the following general cut points for assessing model fit: TLI and CFI close to 0.95, SRMR close to 0.08, and RMSEA close to 0.06. For TLI and CFI, bigger values indicate a better fit, whereas the opposite is the case for RMSEA and SRMR. Others argue for less strict cut points, e.g. CFI and TLI ≥ 0.90, and RMSEA ≤ 0.08 (Brown 2015, Pendergast et al. 2017). For multilevel models, Mplus calculates an SRMR value separately for the within (SRMR-W) and the between level (SRMR-B) of the model. At the between level, the cut point of 0.08 is generally considered to be too strict (Asparouhov and Muthén 2018). Additionally, simulations have shown that SRMR-B is sensitive to large ICC 1 values (above 0.1) and small sample sizes even under optimal simulated conditions, and therefore this measure should be used with caution in applied ML-CFA (Ene 2020).

Table 3. Final model statistics.

Observations	Chi-square	CFI	SRMR-W
136,281	10,274.91	0.976	0.002
Clusters	Degrees of freedom	TLI	SRMR-B
8,128	74	0.941	0.105
Average cluster size	Chi-square p-value	RMSEA
16.77	0.000	0.032

CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, TLI: Tucker Lewis Index, SRMR: Standardised Root Mean Square Residual.

In sum, the model shows good fit by these metrics. For the Chi-square test, the p-value is highly significant, which is not ideal. However, this is most likely due to the sensitivity of the Chi-square fit statistic to large samples. CFA analysis generally requires large samples, which makes Chi-square a poor measure of fit (Pendergast et al. 2017). In this case, with a sample size of 136,281 teachers in 8,128 schools, the significant Chi-square is disregarded.

Model fit for individual countries/economies and Measurement invariance

Although the ambition in this article is to provide a global measure of PLCs in the pooled data, the model has been tested on each of the participants in TALIS 2018 separately. This step is taken to identify participants where the overall model cannot be identified, in order exclude these from the factor analysis, inspired by the approach taken in creating the official TALIS scales (OECD 2019). Belgium, France, Hungary, Mexico and Saudi Arabia are excluded from the analysis on this basis. For France and Hungary, at least one of the questions was not administered. For the others, the model is not identified. The fit statistics for the 42 participants where the model is successfully identified is presented in Appendix 2. It should be noted, that modelling the construct using only the data from specific participants in most cases results in warnings related to non-positive definite matrices, which is a common issue in factor analysis that can be causes by a number of things, for example inadequate sample size, very high correlations, non-normality or model misspecification (Chen et al. 2001, Brown 2015). Researchers with an interest in modelling the PLC measure for specific contexts, instead of using the pooled version, should assess the output and make adjustments if necessary. As an example, the warning of a non-positive definite matrix for the Chile is due to two items having small negative residual variances. Fixing these residuals at zero solves the issue, and model fit is not changed substantially. Similar adjustments was necessary for modelling the official TALIS scales at the participant level (OECD 2019, p. 217).

Measurement invariance

The extent to which constructs are comparable across different populations is often evaluated using the measurement invariance approach. Frequently, three stages of invariance are considered: configural (identical structure), metric (identical structure and equal factor loadings), and scalar (identical structure, factor loadings and intercepts). In the case of large-scale international surveys, Rutkowski and Svetina (2014) recommend evaluating the invariance of constructs by changes in CFI and RMSEA, and suggest cut points of −0.020 and 0.030 for change in CFI and RMSEA, respectively.

It is not currently possible to test the invariance of ML-CFA models using MLR and/or second-order factors in Mplus. To provide some evidence of the invariance, the three constructs CPL, SVR and SC are tested simultaneously for measurement invariance in a single-level model across the 42 countries/economies included in the final model. The results of this analysis are presented in Appendix 3, showing that metric invariance is achieved (ΔCFI = −0.020, Δ RMSEA = 0.003). In comparison, eight of the official teacher scales in the data achieve configural invariance, 22 achieve metric invariance, and only one achieves scalar invariance. For cross-country comparisons, metric invariance implies that it is reasonable to compare strengths of associations between countries, but not scale means (OECD 2019, p. 213–215).

Example use: PLCs and teacher job satisfaction in the Nordic countries

The following section will provide a brief example of how the developed measure of PLCs may be useful for research in different contexts, by investigating the relationship between PLCs and teacher job satisfaction at the school level in the Nordic countries participating in TALIS 2018. Engaging in PLCs have been associated with higher levels of teacher job satisfaction theoretically and qualitatively (Louis 1994, Hord 1997), while few studies have investigated this relationship quantitatively, and results have been inconsistent, as has the measures of both the PLCs and job satisfaction (Zhang et al. 2020).

The official TALIS data includes a composite measure of teacher job satisfaction in the school data (t3pjobsa), which will be used as the dependent variable in this analysis. Previous research related to teacher job satisfaction in TALIS has varied in relation to control variables (Gil-Flores 2017, Liu et al. 2021). This example includes a stepwise model, where Model 1 is without controls, and Model 2 includes controls for school location,¹ school type (public/private) and percentage of students coming from low socioeconomic homes Table 4). The school- and replicate weights are applied in the analysis, using the EdSurvey package in R, to account for the complex survey design (Bailey et al. 2020).

Table 4. Stepwise regression on teacher job satisfaction.

		Model 1			Model 2
	Variable	Coef		SE	Coef		SE
Denmark (N = 128)	PLC	2.50		1.70	2.38		2.06
	Private school				0.22		0.49
	School location: Small town				0.30		0.47
	School location: Town				0.36		0.52
	School location: City				0.37		0.52
	1 to 10% of students low SES				−0.42		0.88
	11 to 30% of students low SES				0.72		0.86
	31% or more are low SES students				−1.10		1.25
	R2	0.016			0.114
Norway (N = 150)	PLC	9.47***		1.82	9.73***		2.02
	Private school				0.59		0.44
	School location: Small town				−0.35		0.41
	School location: Town				−0.70		0.48
	School location: City				−0.66		0.45
	1 to 10% of students low SES				−0.13		0.65
	11 to 30% of students low SES				0.01		0.69
	31% or more are low SES students				1.07		1.04
	R2	0.150			0.178
Sweden (N = 150)	PLC	3.54		2.40	6.34***		1.75
	Private school				−1.35*		0.53
	School location: Small town				0.24		0.81
	School location: Town				0.76		0.64
	School location: City				1.80*		0.76
	1 to 10% of students low SES				−0.33		0.37
	11 to 30% of students low SES				0.27		0.50
	31% or more are low SES students				−0.81		0.74
	R2	0.024			0.180
Finland (N = 147)	PLC	2.84		2.53	1.94		2.55
	Private school				0.15		0.81
	School location: Small town				0.62		0.62
	School location: Town				0.67		0.57
	School location: City				1.05		0.74
	1 to 10% of students low SES				−0.57		0.49
	11 to 30% of students low SES				−0.63		0.63
	31% or more are low SES students				0.77		0.67
	R2	0.014			0.054

Notes: Significance codes *: p ≤ 0.05 **: p ≤ 0.01 ***: p ≤ 0.001.

Reference category for school location ‘A village, hamlet or rural area’.

Reference category for student SES: ‘None’

Among these four Nordic countries, Model 1 shows that only for Norway there is a statistically significant relationship between PLC and teacher job satisfaction, while there is a positive but not significant relationship in the three other countries. When controlling for school location, type and student composition, we also find a statistically significant relationship in Sweden in addition to Norway. This analysis suggests that the strength of the association between these two latent constructs, varies significantly even among relatively similar education systems. While this analysis has served as an example utility of the scale, the relationship between PLCs and teacher job satisfaction calls for further investigation, potentially using more advanced models such as hierarchical linear models or structural equation models. Various school and/or country level contextual factors (e.g. school climate, leadership, centralisation, school/teacher autonomy) might also be included in similar research as mediating and moderating variables to find out their roles in such relations in different countries.

Discussion

This section will discuss the strengths and limitations of the developed measure of PLCs in terms of the alignment with theory and for the use of this measure in secondary research. Additionally, potentials for the further development of such a measure in TALIS are discussed.

Initially, each of the three constructs; collaborative practice with focus on student learning (CPL), shared vision and responsibilities (SVR), supportive conditions (SC), and the overall measure of PLCs are evaluated against the theoretical concept of PLCs. Comparing the indicators in the (CPL) construct with the key elements of PLCs, it is apparent that the CPL construct overlaps more than one of these elements. Specifically, it is theorised that a strong focus on individual student learning and collaborating and reducing teacher isolation can be measured by this construct, while shared responsibility for student learning is measured partly by CPL and partly by SVR. Initially, attempts were made to measure the focus on individual student learning as a separate construct, but this model did not fit the data as expected, and therefore the dimensions were merged. The CPL construct contains indicators of important aspects of each of the elements, such as peer observation and feedback, engaging in discussions about the learning of individual students, collaboration related to evaluating student learning, and involvement in collaborative professional learning. These individual items are associated with important aspects of the practical behaviour that teachers who are part of a PLC should be engaged in, and the answer categories reveal how often such behaviour takes place, enabling the construct to describe the extent to which the individual school is functioning as a PLC, as reported by the teachers. While the CPL construct does include indicators of collaboration related to student assessment, it does not explicitly mention the use of data in assessment, which is a limitation in relation to the theoretical framework.

The SVR construct relates to the elements known as shared vision and shared responsibility for student learning. As described in the theory section, schools operating as PLCs should have a shared vision, and the teachers should be involved in defining this. Likewise, the responsibility for the learning development of individual students is supposed to be a shared responsibility of the teachers. The indicators in the SVR construct measure the extent to which the teachers agree that they share common beliefs about teaching and learning, and whether or not the teachers have opportunities to participate in school decisions. The SVR construct also includes information about the existence of a collaborative culture with mutual support and shared responsibility for school issues. In PLC literature, the responsibility for student learning is shared by the teachers, and it is a limitation that the item included is not directed at this issue more specifically. Arguably, the indicators in CPL and SVR together capture the notion of which teachers collaborate to resolve such difficulties, so this aspect is present in the overall measure of PLCs.

The final construct, SC, measures the conditions for (and priority of) professional development in the school, as perceived by the teachers. If the school principal and management do not support and dedicate time for the teachers to participate in professional development, as indicated by the items in this construct, PLCs are highly unlikely to be successful. The indicators in this construct do not address specific types of professional learning, but agreement on these indicators will translate into professional development being prioritised by the school principal or management, thereby making the engagement in PLCs, as it is measured by CPL, more likely.

Researchers interested in the concept of PLCs, and in testing or validating hypotheses across a wide variety of national contexts, may find this measure useful. The concept of PLCs is popular world-wide, and the theoretical assumptions and expectations of positive outcomes seems to be considered as universal. This universal nature of PLCs has been difficult to assess given that quantitative research so far has used widely different measures of PLCs and data. The example analysis of job satisfaction in this paper suggest that there is reason to be sceptical about the universal and positive effects of PLCs across different contexts. The developed measure along with the extensive teacher- and school data available in TALIS provides an opportunity conduct comparable research across a wide range of national contexts.

When applying this measure, researchers should be explicit about strengths, limitations and the overlap between the individual constructs in relation to the PLC literature, on the basis of the discussion above. Some evidence of the invariance of the measure is provided; and while there are methodological limitations, the metric invariance of the items included suggests the feasibility of comparisons of statistical associations across participants in TALIS 2018.

Given the significance of the concept of PLCs internationally, future cycles of TALIS could pursue the option of developing an official measure of PLCs in the data. Based on the development of this measure and the review of items included in the questionnaire, it can be argued that items measuring the use of data in student assessment, and the specific ways in which teachers collaborate to improve teaching practice, would be particularly useful in improving the chance of creating an even more comprehensive measure of PLCs at school level in TALIS.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed here

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was supported by The Carlsberg Foundation under grant CF19-0751.

Notes on contributors

Anders Astrup Christensen

Anders Astrup Christensen is currently a Ph.D. fellow at the department of Educational Sociology, Danish School of Education at Aarhus University, Denmark, and is primarily interested in school context, teacher practice, student outcomes and inequality.

Notes

1. The categories ‘City’ and ‘Large city’ has been collapsed to ‘City’ because ‘Large city’ was unused in Finland and Norway.

Appendices

Table 1. Full list of variables considered in initial selection.

Name	Text
	18. Approximately how many 60-minute hours did you spend on the following tasks during your most recent complete calendar week, in your job at this school? (Hours/week)
tt3g18b	Team work and dialogue with colleagues within this school
tt3g18g	Professional development activities
	22. During the last 12 months, did you participate in any of the following professional development activities? (Yes/no)
tt3g22g	Peer and/or self-observation and coaching as part of a formal school arrangement
tt3g22h	Participation in a network of teachers formed specifically for the professional development of teachers
	23. Were any of the topics listed below included in your professional development activities during the last 12 months? (Yes/no)
tt3g23d	Student assessment practices
tt3g23h	Approaches to individualised learning
tt3g23l	Analysis and use of student assessments
	28. How strongly do you agree or disagree that the following present barriers to your participation in professional development? (Strongly disagree, Disagree, Agree, Strongly agree)
tt3g28c	There is a lack of employer support
tt3g28d	Professional development conflicts with my work schedule
tt3g28f	There is no relevant professional development offered
tt3g28g	There are no incentives for participation in professional development
	29. In this school, who uses the following types of information to provide feedback to you? (2: Box checked/not checked: Principal or member(s) of management team, 3: Box checked/not checked: Other colleagues within the school)
tt3g29a2	Observation of my classroom teaching
tt3g29f2	Self-assessment of my work (e.g. presentation of a portfolio assessment, analysis of my teaching using video)
tt3g29a3	Observation of my classroom teaching
tt3g29e3	School-based and classroom-based results (e.g. performance results, project results, test scores)
tt3g29f3	Self-assessment of my work (e.g. presentation of a portfolio assessment, analysis of my teaching using video)
	32.Thinking about the teachers in this school, how strongly do you agree or disagree with the following statements? (Strongly disagree, Disagree, Agree, Strongly agree)
tt3g32a	Most teachers in this school strive to develop new ideas for teaching and learning
tt3g32b	Most teachers in this school are open to change
tt3g32c	Most teachers in this school search for new ways to solve problems
tt3g32d	Most teachers in this school provide practical support to each other for the application of new ideas
	(Continued) 33.On average, how often do you do the following in this school? (Never, Once a year or less, 2–4 times a year, 5–10 times a year, 1–3 times a month, Once a week or more)
tt3g33a	Teach jointly as a team in the same class
tt3g33b	Observe other teachers’ classes and provide feedback
tt3g33d	Exchange or develop teaching materials with colleagues
tt3g33e	Engage in discussions about the learning development of specific students
tt3g33f	Work with other teachers in this school to ensure common standards in evaluations for assessing student progress
tt3g33h	Take part in collaborative professional learning
	48. How strongly do you agree or disagree with these statements, as applied to this school? (Strongly disagree, Disagree, Agree, Strongly agree)
tt3g48a	This school provides staff with opportunities to actively participate in school decisions
tt3g48d	This school has a culture of shared responsibility for school issues
tt3g48e	There is a collaborative school culture which is characterised by mutual support
tt3g48f	The school staff share a common set of beliefs about teaching and learning
tt3g48h	This school encourages staff to lead new initiatives
	49.How strongly do you agree or disagree with the following statements about what happens in this school? (Strongly disagree, Disagree, Agree, Strongly agree)
tt3g49d	If a student needs extra assistance, the school provides it.

 

Table 2. Individual country/economy analysis of the final model.

Country/economy	N.obs	CFI	TLI	RMSEA	SRMR-W	SRMR-B
Argentina (Buenos Aires)	2078	0.977	0.944	0.034	0.004	0.167
Australia	3320	0.981	0.953	0.030	0.005	0.147
Austria	4188	0.987	0.969	0.023	0.005	0.178
Belgium (Flemish and French Communities)	5126	0.971	0.928	0.031	0.008	0.227
Brazil	2405	0.982	0.955	0.034	0.006	0.150
Bulgaria	2847	0.99	0.975	0.021	0.004	0.118
Canada (Alberta)	1005	0.988	0.972	0.025	0.005	0.147
Chile	1952	0.988	0.971	0.026	0.007	0.133
China (People’s Republic of) (Shanghai)	3975	0.997	0.993	0.016	0.002	0.077
Chinese Taipei	3825	0.992	0.980	0.020	0.003	0.257
Colombia	2381	0.986	0.966	0.028	0.004	0.153
Croatia	3311	0.991	0.978	0.021	0.003	0.156
Cyprus	1586	0.966	0.916	0.048	0.005	0.218
Czech Republic	3419	0.990	0.976	0.020	0.003	0.135
Denmark	1927	0.977	0.943	0.030	0.005	0.140
England	2214	0.997	0.992	0.013	0.003	0.144
Finland	2833	0.987	0.968	0.023	0.003	0.143
Georgia	3069	0.991	0.978	0.022	0.002	0.088
Israel	2409	0.993	0.983	0.019	0.005	0.172
Italy	3554	0.985	0.964	0.026	0.004	0.173
Japan	3539	0.984	0.961	0.026	0.004	0.247
Kazakhstan	6564	0.980	0.952	0.028	0.004	0.213
Korea	2908	0.987	0.968	0.030	0.003	0.177
Latvia	2251	0.988	0.971	0.024	0.003	0.160
Lithuania	3750	0.995	0.988	0.017	0.003	0.142
Malta	1600	0.975	0.939	0.037	0.005	0.293
Netherlands	1785	0.990	0.975	0.022	0.003	0.115
New Zealand	2124	0.992	0.980	0.019	0.004	0.132
Norway	4016	0.971	0.928	0.030	0.003	0.175
Portugal	3616	0.982	0.956	0.029	0.003	0.142
Romania	3639	0.996	0.989	0.015	0.002	0.120
Russia	4005	0.994	0.984	0.018	0.003	0.088
Singapore	3265	0.952	0.882	0.050	0.002	0.149
Slovakia	2987	0.985	0.963	0.024	0.004	0.224
Slovenia	2062	0.991	0.978	0.019	0.003	0.172
South Africa	2034	0.960	0.901	0.048	0.012	0.258
Spain	7358	0.958	0.897	0.043	0.006	0.213
Sweden	2607	0.987	0.967	0.024	0.004	0.136
Turkey	3928	0.991	0.978	0.028	0.003	0.135
United Arab Emirates	8545	0.992	0.980	0.023	0.010	0.277
United States	2450	0.990	0.974	0.024	0.004	0.135
Viet Nam	3824	0.995	0.987	0.012	0.002	0.085

 

Table 3. Results of measurement invariance test, single level model of the constructs CPL, SVR and SC.

Model	Chi-sq (df)	CFI	Δ CFI	TLI	RMSEA	Δ RMSEA	SRMR
Configural	36,578.37 (3,182)	0.927	-	0.910	0.057	-	0.044
Metric	46,369.87 (3,644)	0.907	−0.02	0.900	0.06	0.003	0.065
Scalar	173,211.79 (4,106)	0.630	−0.277	0.647	0.113	0.053	0.195