Professional learning networks: a conceptual model and research opportunities

ABSTRACT

Background

Professional Learning Networks (PLNs) of educators represent a promising approach to achieving continuous school improvement. At the same time, a range of studies in this area, including several systematic reviews and meta-analyses, report multiple conceptual and methodological challenges, or, at best, mixed results.

Purpose and sources

The aim of this theoretical discussion paper, therefore, is to seek, first, to synthesise and connect to previous studies focused on professional learning communities and networks by combining and reflecting on their findings and recommendations. Second, we aim to contribute to the methodological development of the field in order to propose research that can link what happens in PLNs to changes in outcomes for students. For the latter, we will also make use of new insights from the field with regard to the use of big data in education.

Main argument

We propose a conceptual model of what defines PLNs, enactment process variables, and influencing factors, presenting our theory-of-action for how PLNs can be effective. Second, we discuss challenges and recommendations in studying PLN impact regarding research approach, research design and measurement. This discussion includes consideration of the use of big data to help to make the analysis of patterns in, and relations between, different types of PLN research data more efficient and reliable.

Conclusion

We need to define and study the processes and effects of PLNs more efficiently and effectively, to support PLNs in fulfilling the promise of increased teacher learning, improved outcomes for students, and, ultimately, sustainable school improvement at scale.

KEYWORDS:

• Professional learning networks (PLNs)
• professional development
• methodology
• school improvement
• professional collaboration
• conceptual model

Introduction

To improve teacher and student learning, there is a growing focus on the collaborative power of inter-school social capital (Brown and Poortman 2018). Essential to effective teacher professional development is the understanding that improvement should be grounded in the structured learning of teachers, resulting in positive changes to their practice as well as the learning of their students (Borko 2004; Darling-Hammond, Hyler, and Gardner 2017; Desimone 2009; Van Veen et al. 2010). Different systematic reviews in this area report that characteristics, such as active learning in collaboration, are, over a longer period of time, essential features of effective professional development (Borko 2004; Darling-Hammond, Hyler, and Gardner 2017; Van Veen et al. 2010). Moreover, these reviews explicitly state that strong professional communities can foster teacher learning. This perspective aligns with both the raison d’être and the internal processes of Professional Learning Networks (PLNs).

PLNs, such as research learning networks (Brown and Flood 2019), teacher design teams (Binkhorst, Poortman, and Van Joolingen 2017; Binkhorst 2017; Kali, McKenney, and Sagy 2015), and data teams (Poortman and Schildkamp 2016; Wayman, Midgley, and Stringfield 2006), engage in stages such as knowledge sharing, knowledge creation, the development of new practices and the joint trial and refinement of these practices. The success of all of these stages depends entirely on effective collaborative efforts between teachers. These processes have also been shown to have an impact on not only the learning of teachers in the PLNs but also the learning of the wider community of colleagues to which they belong (e.g. Rose et al. 2017). Professional Learning Networks (PLNs) of teachers ‘committed to and capable of creating deep and broad teaching and learning’ (Hargreaves and Shirley 2009, 107) therefore represent a promising approach to achieving continuous school improvement (Sahlberg 2011). At the same time, there is still a lot to learn about the process of professional collaboration and the conditions that influence this collaboration in PLNs (Gore, Lloyd, Smith, Bowe, Ellis and Lubans; Brown and Poortman 2018; Doğan and Adams 2018; Vangrieken et al. 2017; Warren Little 1990).

Professional Learning Networks (PLNs) can be described as ‘any group who engage in collaborative learning with others outside of their everyday community of practice in order to improve teaching and learning’ (Brown and Poortman 2018, 1). This is a broad description, encompassing a large range of network types, both within-school and across schools. Over the past few decades, a range of studies, including several systematic reviews, have addressed the effects of educators learning collaboratively in Professional Learning Communities and networks, as well as the factors influencing this success. Although these studies conclude that the evidence of participation of educators is promising (Doğan and Adams 2018; Gast, Schildkamp, and van der Veen 2017; Lomos, Hofman, and Bosker 2011; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008), they also report conceptual and methodological challenges. For example, Vescio, Ross, and Adams (2008) conclude that only a few studies have looked at the impact of PLCs on teacher practice or student learning. Lomos, Hofman, and Bosker (2011), meanwhile, report a significant but small effect, although their work was based on only five studies. Lomos et al. concluded that further research should focus both on a clear conceptualisation and empirical validation of the key dimensions. More recently, Doğan and Adams (2018) reached the same conclusions, raising concerns about the design and further methodological issues in research, as well as issues regarding conceptualisation (see also Daly and Stoll 2018). We found it striking that reviews from more than a decade ago, as well as much more recent ones, came to these same conclusions. While more and more educators are participating in network initiatives, the field needs to advance in showing more univocal impact on student outcomes, by addressing the conceptual and methodological challenges brought forward in the literature.

Purpose

The aim of this theoretical discussion paper, therefore, is first to seek to synthesise and connect to previous studies focused on professional learning communities and networks by combining, and reflecting on, their findings and recommendations. The aim here is to specify more explicitly the vital components of a PLN. In other words, we are seeking to understand what the essential characteristics of a PLN are. Our intention is to do this by critically analysing and contrasting PLN descriptions in the key literature referred to above, making a distinction between what defines a PLN, what is part of the enactment process of a PLN and what influences a PLN. Second, once this clarity is achieved, we seek to contribute to the methodological development of the field, by proposing research that can link what happens in PLNs to changes in outcomes for students. For the latter, we will also make use of new insights from the field regarding the use of big data in education.

Conceptual clarity in PLN research

PLN definition

Various forms of collaborative learning of educators in communities and networks are often included in the literature, but not always explicitly distinguished. These are Professional Learning Communities (PLCs), which can be whole-school PLCs, where the school aims to function as a PLC (DuFour 2007; Vanblaere and Devos 2016; Stoll et al. 2006); or networked, whole-school PLCs, where groups of schools work together to share resources and/or enhance the quality of professional learning and the capacity for continuous improvement (e.g. Katz and Earl 2010; Díaz-Gibson et al. 2017; Muijs 2015) – these are also called Networked Learning Communities.

In most of the literature, the term PLC is broadly defined to include different types of networks (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008). According to Stoll et al. (2006, 242), ‘Networked learning communities and PLCs rest on similar assumptions about how teachers learn and change their practice’. Doğan and Adams (2018, 638) report, for example, ‘We accepted any type of PLCs without concentrating on its structure and context’ and included school-based PLCs, cross-school PLCs, and within-school within-subject PLCs.

The focus of our paper is on the type of networks where several educators from different schools (or departments) come together with a particular purpose related to outcomes for students. These networks could be (cross-school) teacher design teams, research learning networks, research–practice partnerships, or data teams. The core of our definition is that a PLN consists of a group of educators (e.g. teachers, school leaders, possibly in collaboration with researchers, and/or policy-makers) coming together with others, outside of their everyday community of practice, with the intention of engaging in collaborative learning to improve teaching, and their students’ learning. Student learning outcomes do not necessarily have to equate to achievement outcomes, however; PLNs might also focus on issues of wellbeing or equity or on engaging in a critical examination of the purpose and the aims of the curriculum (Brown 2020; Datnow and Park 2018).

The essence of the definition of Professional Learning Communities (PLCs) (not limited to, but including, our concept of PLNs) is largely agreed upon in the literature (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011; Sleegers et al. 2013; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008). At the same time, however, many authors conclude of PLCs that ‘People (…) describe every imaginable combination of individuals with an interest in education (…) so ubiquitously that it is in danger of losing all meaning’ (DuFour 2004, 6); ‘there is no universal definition’ (Stoll et al. 2006, 222); ‘is difficult to grasp’ (Lomos, Hofman, and Bosker 2011, 122); and ‘a meaningless label’ (Vangrieken et al. 2017, 49). This critique highlights that, if PLNs are to provide a useful tool for improving teacher and student outcomes, then there is a need to further define such a concept (Daly and Stoll 2018). Several systematic review studies in this area address the issue of conceptualisation (Lomos, Hofman, and Bosker 2011; Doğan and Adams 2018; Gast, Schildkamp, and van der Veen 2017; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008), partly using different labels for the same variables and the same labels for different variables. For example, Stoll et al. (2006) suggest the following five key characteristics or features (226–227): (1) ‘shared values and vision’; (2) ‘collective responsibility’; (3) ‘reflective professional inquiry’; (4) ‘collaboration’; and (5) promotion of group, as well as individual, learning.

They also include other factors that can help or hinder the creation and development of PLNs, such as school context and specifically the active support of leadership at all levels. Lomos, Hofman, and Bosker (2011, 139) speak of ‘the concept’s key dimensions’ (also called ‘five interconnected variables’ (124) further on in the text), which to some extent overlap with the five key characteristics of Stoll et al. (2006): namely, (1) reflective dialogue; (2) deprivatization of practice or feedback on instruction; (3) collaborative activity; (4) shared sense of purpose and (5) a collective focus on student learning. Other studies refer also to ‘essential characteristics’ (Lomos, Hofman, and Bosker 2011, 139; Vescio, Ross, and Adams 2008); ‘influencing factors’ (Prenger, Poortman, and Handelzalts 2017, 77); ‘features/essential features’ (Rincón-Gallardo and Fullan 2016, 5); ‘dimensions’ (Sleegers et al. 2013, 118); ‘core principles’ (DuFour 2004, 1); ‘defining characteristics’ (Vangrieken et al. 2017, 49); ‘key conditions’(Brown and Poortman 2018) or ‘conditions for success’ (Vangrieken et al. 2017, 47).

In calling variables such as shared values and vision or reflective dialogue a ‘key characteristic’ or ‘defining characteristic’, we need to consider what the essence of a PLN is, as well as its teleology. For example, such statements raise the question of whether educators coming together with the intention of engaging in collaborative learning to improve teaching and their students’ outcomes cannot be defined as a PLN unless they also have shared visions and values at the start of the process. Or, whether they are not a PLN if they do not yet, or never will, engage in reflective professional inquiry. One of Rincón-Gallardo and Fullan’s (2016) essential features, for example, includes the word ‘developing’ (strong relationships of trust and internal accountability), which also indicates that such variables are not dichotomous (i.e. they either exist or they do not) and that PLNs should always be moving towards their end purpose of improved teacher and student outcomes. Vangrieken et al. (2017, 55) also noted that the extent to which teacher communities meet the theoretically proposed criteria can be described as their degree of maturity. Stoll et al. (2006) found that PLNs are ‘fluid, rather than fixed, entities’ (228) and may develop through different stages over time.

Consequently, terms such as ‘core principles’ or ‘key features’ appear partly to refer to what defines a PLN, and partly to refer to what influences a PLN. To clarify the concept of PLNs further, therefore, we need to distinguish clearly between defining components on the one hand and influencing factors on the other hand. In addition, however, Stoll et al.’s (2006) notion of development, Vangrieken et al.’s (2017) suggestion concerning ‘the extent to which teacher communities meet the theoretically proposed criteria’ (55) and Rincón-Gallardo and Fullan's (2016) inclusion of the word ‘developing’ in one of their variables, point to a third type of characteristic, which is part of the enactment process of the PLN’s intentions.

PLN enactment process

PLN enactment process variables are variables that are assumed to influence PLN outcomes. For example, active engagement in reflective dialogue by PLN participants will increase the likelihood that the PLN will result in teacher learning, changing teacher practice and, ultimately, student learning. Based on the literature, we identified the following enactment variables. First, several reviews include collaboration as what we would call an enactment process variable (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008). Second, notions related to a focus on students and a shared sense of purpose or shared values and vision (e.g. Lomos, Hofman, and Bosker 2011, 124; Prenger, Poortman, and Handelzalts 2017; Stoll et al. 2006, 226) included in various studies can all be categorised as shared sense of purpose focused on student learning. Third, reflective professional inquiry can be identified as an enactment process variable. Other terms that were used in the literature here include reflective dialogue, active learning strategies, and continuous teacher learning (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011; Stoll et al. 2006; Vangrieken et al. 2017; Vescio, Ross, and Adams 2008). These all relate to group as well as individual learning, reflective dialogue, deprivatization of practice and applying new ideas from conversations, data, and literature to problem-solving in the PLN (Stoll et al. 2006; Meijlof 2018). Fourth, PLN leadership has been identified as another enactment process variable (Doğan and Adams 2018; Vangrieken et al. 2017). We consider it important to include PLN leadership and distinguish this from leadership influencing the PLN from outside, i.e. school leadership (Brown 2020). Finally, to share and further develop knowledge developed within the PLN with other colleagues in the participating schools and other institutions outside the PLN, PLN participants need to cross the PLN boundary (Akkerman and Bakker 2011) to make sure teaching practice and outcomes for students are improved for all students in the network. Individuals crossing the ‘boundaries’ between PLN and their school/colleagues are called ‘knowledge brokers’ (Poortman and Brown 2018; Farley-Ripple and Grajeda 2019), with the notion of brokerage comprising the process of communicating innovation from one community to another such that it engenders changes in that community’s understanding and their actions (Brown 2020; Rogers 1995).

Factors influencing the PLN enactment process

The individual, school and policy context factors influencing the PLN enactment process can be distinguished at different levels. Firstly, at the micro level, we have individual PLN participants’ knowledge, skills and attitudes (Stoll et al. 2006; Prenger, Poortman, and Handelzalts 2019). Secondly, at the meso level, we have PLN size and composition (Decuyper, Dochy, and Van den Bossche 2010; Prenger, Poortman, and Handelzalts 2017), network leadership and school(s) context(s) (Brown 2020; Katz and Earl 2010; Stoll et al. 2006; Lomos, Hofman, and Bosker 2011). Third, at the macro level, we have the broader policy and accountability context(s) (Chapman 2014; Katz and Earl 2010; Stoll et al. 2006; Prenger, Poortman, and Handelzalts 2021). Each of these levels will now be discussed further.

Individual participants’ knowledge, skills and attitudes refer to individuals’ prior knowledge about the PLN theme and motivation. A range of factors such as trust, self-efficacy, and confidence are included in studies about the relation with PLN impact or with educational innovation in general (Decuyper, Dochy, and Van den Bossche 2010; Prenger, Poortman, and Handelzalts 2017). PLN size and composition can influence these individual factors because a variation in prior knowledge and motivation can work to compensate for a lack of these in individuals. Schildkamp, Smit, and Blossing 2019, for example, found that ‘Differences between team members can be stimulating if the team has discussions that lead to cognitive conflict; that is, differences in opinion, premises, and ideas’ (409). These kinds of conflict, if addressed properly in a team, can stimulate learning (Achinstein 2002; Butler and Schnellert 2012; Katz, Earl, and Ben Jaafar 2009). Finally, trust, as well as consensus regarding the purpose of the network might decrease as a function of size; while the time, effort and skill required to coordinate the network increases (Ehren and Godfrey 2017).

The school context concerns factors such as leadership and facilitation (Earley and Greany 2017; OECD 2016; Stoll 1999). To get the most out of engaging with PLNs, school leaders must first understand their role as instructional leaders and the impact this role can have. Here, it is worth recalling the work of Robinson (e.g. Robinson, Hohepa, and Lloyd 2009), which demonstrates that instructional leadership approaches result in the most substantial benefits for student outcomes. In particular, Robinson, Hohepa, and Lloyd (2009) suggest that the act of school leadership with the biggest single impact is ‘promoting and participating in teacher learning and development’ (2), which they indicate has an effect size of 0.84. As well as having an instructional focus, however, school leaders must also lead ethically, with a commitment to social justice and doing the best for each child (Warren Little 1990). An ethical instructional approach, though, is just one prerequisite for schools to engage effectively with PLNs: school leaders must also want to reach out beyond the boundaries of their schools and wish for their teachers to engage in collaborative endeavours with others (Armstrong 2015; Azorín, Harris, and Jones 2020).

Education policy refers to ‘the actions taken by governments in relation to educational practices, and how governments address the production and delivery of education in a given system’ (Viennet and Pont 2017, 19). Work pressure and staff turnover, in an age of both accountability and continuous improvement, hinder teachers from feeling that there is room to experiment with what was learnt in their PLN, as well as possibly impede their participation and reflective professional inquiry in the first place (Brown 2020; Prenger, Poortman, and Handelzalts 2021). If educators are expected to engage in continuous self-improvement, they should have the opportunity to spend time and resources on PLN participation as well as experimenting with applying what was learnt with colleagues in their schools, in the immediate and long term.

A proposed conceptual model

In this section, we propose and discuss a conceptual model that can link what happens in PLNs to changes in outcomes for students. Figure 1¹ shows our conceptual model, which elaborates the PLN concept in relation to three key areas: (1) what defines PLNs, (2) the enactment process variables, and (3) influencing factors at different levels (A). We have included arrows from the influencing factors to the enactment process, (B) teacher learning and changed practice – in and outside the PLN, and ultimately (C) outcomes for students and (D) sustainable school improvement (at scale), showing our theory-of-action for how PLNs can be effective. This model can help us further investigate the link between educators engaging in PLNs and outcomes for students (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011), a challenge addressed in this part of our paper.

Figure 1. PLN conceptual model.

Source: This figure is the authors’ unpublished, original work created for this publication.

Linking PLN processes to outcomes for students

Challenges in PLN research

According to Doğan and Adams (2018), the ultimate aim of PLCs is to improve outcomes for students. Relating teacher practice to outcomes for students in this area, however, is challenging and needs further research. The review studies we consulted all come to this conclusion: much more rigorous research is needed, in order to build evidence supporting the impact on teaching practice and outcomes for students (Doğan and Adams 2018; Lomos, Hofman, and Bosker 2011; Stoll et al. 2006; Vescio, Ross, and Adams 2008; Vangrieken et al. 2017). According to Vescio, Ross, and Adams (2008), a range of further studies are needed, including quantitative, qualitative and longitudinal studies, concerning both changes in teacher practice and student achievement: ‘Although, the analysis of data about student achievement is time-consuming, it is essential … ’ (90) (see also Wells and Feun 2013). At the same time, they do not only refer to the often narrow range of learning captured using achievement tests: teachers participating in PLNs could also collect data about their students’ learning, using case studies that examine changes in student work over time. A few years on from 2008, Lomos, Hofman, and Bosker (2011, 139) raised the ‘clear need for more empirical studies’. Specifically, they were referring to the need for more multilevel studies that are required to address the teacher and school-level characteristics (Bosker and Scheerens 1994; Sleegers, Geijsel, and Van den Berg 2002) – in line with our identification of different levels of influencing factors (see also Figure 1) – and to specify, more clearly, the network activities. In addition, they call for more longitudinal studies and a cognitive achievement measure that offers insight into student progress rather than producing one single test result (Lomos, Hofman, and Bosker 2011, referring to Hofman and Dijkstra 2010). Seven years after this study, Doğan and Adams (2018) concluded that five PLC features (facilitator/leader support; collaboration; active learning strategies; focus on instruction and students; and reflective dialogue) were associated with improvements in teacher practices. According to Doğan and Adams’ (2018) review, all features except reflective dialogue are directly related to increases in student achievement. At the same time, Doğan and Adams (2018) did not have evidence to relate teacher practice to student achievement. They concluded that this is an area for further research. Several other studies, specifically focused on PLNs as we define them in this paper, also concluded that we need to look further into both depth and breadth of learning, and the connection between networks, changes in practice and student learning (Daly and Stoll 2018; Gore et al. 2015).

Apart from the issues related to the type of research needed, in the studies reviewed there are too issues related to reporting on methods, including reliability and validity matters; and regarding strategies for data collection. Issues relating to reporting on data collection, for example, concerned a lack of information on selecting respondents, PLN implementation, the role of the researcher, and strategies to enhance reliability and validity: ‘Use of member checks, triangulation of data, and details on data analysis were sorely lacking in most cases’ (Doğan and Adams 2018, 654). Doğan and Adams (2018) included articles that examined, at least, changes in teacher practice and outcomes for students. They ended up identifying 13 studies, despite having started with 140 peer-reviewed and empirical articles, after initial filtering. Lomos, Hofman, and Bosker (2011) included the validity and reliability of instruments used for data collection as an inclusion criterion for studies they reviewed: they found only five studies meeting their criterion. Vescio, Ross, and Adams (2008) also only included 11 studies, and the Vangrieken et al. (2017) study included 40 studies focused on teacher communities (489 after first filtering), of which PLNs are a subset.

We have highlighted issues with research designs and instruments, the lack of studies reporting on quality criteria and the low number of studies reporting effects as found in the systematic review. These considerations make it clear that much more work, and much more rigorous research is necessary in the areas of research design and measurement in this area to be able to link outcomes for students to educators’ PLN participation. The main challenges in advancing the field centre on demonstrating a more univocal impact on student outcomes. As such, these challenges spotlight matters of research approach, research design, and measurement: we discuss each of these in turn, in the paragraphs below, with reference to our conceptual model (Figure 1).

Research approach

Doğan and Adams' (2018) study raises concerns about how to link PLN processes to PLN outcomes, which is also a general problem for teacher professional development and outcomes for students (Gore et al. 2017). The challenge of linking student achievement (or other types of outcomes for students) to any type of educator professional development starts, as Yoon et al. (2007, 4) assert, with the following line of reasoning:

Professional development affects student achievement through three steps. First, professional development enhances teacher knowledge and skills. Second, better knowledge and skills improve classroom teaching. Third, improved teaching raises student achievement. If one link is weak or missing, better student learning cannot be expected. If a teacher fails to apply new ideas from professional development to classroom instruction, for example, students will not benefit from the teacher’s professional development.

Kirkpatrick (1996), Guskey (1998); Guskey and Sparks (2004), and Desimone (2009) have also referred to logics of action similar to that expressed by Yoon et al. (2007). Specifically, to accomplish improved outcomes for students as a result of PLN participation, educators need to first apply their knowledge and skills regarding the PLN goal; they need to have learnt the knowledge and skills before applying them and need, primarily, to have been satisfied about participating in the intervention (see also Poortman and Schildkamp 2016). However, as Figure 1 shows visually, teachers’ PLN participation may not result in outcomes for students according to such a simple linear pathway. Programme theory, also referred to as theory of action, represents ‘a variety of ways of developing a causal model linking programme inputs and activities to a chain of intended or observed outcomes, and then using this model to guide the evaluation’ (Rogers 2008, 30). Many logic models use single, linear, causal paths. According to Rogers (2008), the question is how appropriate this is for most human service interventions, of which education is one. Simple, linear models risk providing less useful information for replication (see also Brown and Flood 2018; Lortie-Forgues and Inglis 2019). Buccini et al. (2019) discuss theory-driven programme impact pathway (PIP) analysis as a way of explicitly mapping and assessing the mediating steps between the inputs and outcomes of their programme, following a causal logic. Such an analysis, they claim, helps answer not only whether the impact was achieved but also how and why it was achieved – or not – taking into account contextual influencing factors. Yoon et al. (2007) also specified that professional development must be of high quality in its theory of action. Thus, Figure 1 shows how we translated the linear line of reasoning from teacher satisfaction, learning and application resulting from PLN-participation to outcomes for students, to a logic model for a more complicated intervention with different types of variables at different levels (cf. Buccini et al. 2019). We suggest that researchers need to be explicit about their theory of action (or ‘logic model’), define the components well (e.g. using literature reviews to determine and define relevant variables regarding ‘individual participants’ knowledge – skills’, or ‘school context factors’) and use this to guide their evaluation (Buccini et al. 2019), in order to contribute to advancing the field in linking educator participation in PLNs with outcomes for students.

Research design

According to Yoon et al. (2007, 3), apart from the line of reasoning, the ‘other [point] is that the empirical evidence is of high quality – that the study proves what it claims to prove’. At the same time, Yoon et al. (2007) identified a paucity of rigorous studies into the impact of teacher professional development on student achievement (see also Gore et al. 2017). They only include particular randomised controlled trials (RCTs) and particular quasi-experimental studies in their review: nine out of a total of 1,300 initially identified. None of these concerned PLNs (although more recent studies have since emerged; e.g. see Rose et al. 2017). Lortie-Forgues and Inglis (2019), however, argue that although RCTs are widely regarded as the ‘gold standard’ and increasingly prioritised in education internationally, the results are often uninformative. This may be due to the lack of reliability of the literature on which interventions that are part of the RCTs are based. This relates to the issues we addressed in our ‘Challenges in PLN research’ section. Additionally, insights from research on which RCTs were based might not have been adequately translated into effective interventions or implemented successfully. In many PLN studies, the PLN concept, let alone the way it was implemented exactly, lacks a clear definition (Doğan and Adams 2018). The design of the trials themselves may also have been flawed (Lortie-Forgues and Inglis 2019). Cochran-Smith and Fries (2005) argue that RCTs are only appropriate after sufficient theoretical and empirical analysis, to identify competing interventions that reflect the most promising alternatives known to have impact (Gore et al. 2015). Gore et al. (2015) provide a detailed protocol for a cluster randomised controlled trial into the effects of teachers working in professional learning communities, which helps avoid the possible issues with uninformative RCTs in terms of unreliable literature on which interventions are based; inadequate translation of insights into interventions; and flawed design of RCTs (Lortie-Forgues and Inglis 2019). Consequently, in addition to developing and explicitly using a clear theory of action, researchers need to translate their insights carefully from basic research into interventions. These should be tested at a small scale and in a variety of contexts before they are subsequently tested in larger scale (quasi-) experimental studies. Their implementation could also be supported with a detailed and informative intervention protocol.

Measurement and the use of big data

In terms of measurement, further clear conceptualisation of, and consistency regarding, the PLN-elements detailed in Figure 1 is required. For example, what exactly is ‘reflective professional inquiry’? The definitions for ‘reflective dialogue’ by Tam (2015), and Schneider and Kipp (2015), cited in Doğan and Adams (2018), are very different. According to Vangrieken et al. (2015, 21), ‘various, often ill-defined, terms were used’ for ‘collaboration’ in the studies reviewed. This impedes making a clear connection between these variables and outcomes for students over a number of studies.

Other important measurement questions to answer include how can we move beyond singular, self-report methodologies that are also more valid and reliable, but at the same time not more time-consuming for the respondents (Vescio, Ross, and Adams 2008)? Apart from the challenge of tightly defining reflective professional inquiry (i.e. there is a validity issue; cf. Brown et al. 2021), it is difficult to ask respondents about this retrospectively (i.e. there is also a reliability issue), both quantitatively and qualitatively. Although there are many examples of studies into what is going on in collaborative teacher learning meetings (e.g. Horn and Little 2010; Mintrop and Zumpe 2019; Ebbeler et al. 2017), we need to know more about several aspects. These include the link between the process and what teachers really learn from these meetings, how they apply their learning in their classrooms and to what extent, and how this impacts their students in the entire network of participating schools (Butler and Schnellert 2012; Doğan and Adams 2018; Gore et al. 2017; Lai and McNaughton 2018).

There are several ways of approaching this measurement challenge, and these should not be considered separately from the rigorous research design challenge. Different types of data need to be collected in combination and triangulated (Butler and Schnellert 2012; Lai and McNaughton 2018). For example, PLN meetings could be audio- or video-recorded and facilitators could keep minutes of meetings (c.f. Poortman and Schildkamp 2016; Binkhorst 2017). In addition, teacher learning and changed practice could be assessed more specifically, by administering actual observations and assessments measuring the intended learning outcomes (cf. Gore et al. 2017). Products, such as lesson material, guidelines and plans resulting from meetings, could also be part of the data (De Vries and Prenger 2018; Voogt et al. 2011). Different types of student achievement data or other types of outcomes for students – related to the PLN goal – need to be included in data collection and analysis. For example, as stated by Penuel and Shepard (2016), this should not only include achievement on a certain kind of (standardised) assessment but also include learning in a broader sense, and students’ ability to engage in problem-solving and reasoning. Schildkamp (2019) adds that it may also include student data in areas less frequently assessed, such as wellbeing, citizenship, information literacy, and self-regulation. This can additionally include the use of student voice data. Data team PLNs, for example, can collect student achievement data. This might be (final) exam marks, the number of students who fail/pass to subsequent grades, as well as student voice data, for example in the form of questionnaire and interview data about students experiences, to draw conclusions about how to improve student outcomes, including outcomes such as student wellbeing (Poortman and Schildkamp 2016).

One of the earliest reviews referred to in our paper, Vescio, Ross, and Adams (2008), states that the analysis of student achievement data is time-consuming. Therefore, another way to approach this measurement challenge is by using ‘big data’. The increase in the amount of data, the increased availability and accessibility of data in electronic form, and the linking of these data files are defined as ‘big data’ (Veldkamp et al. 2017). These data can be obtained from students or teachers for specific purposes, they can be stored by third parties in administrative systems, and data can be recorded from the interaction of participants with online systems. Big data have the following characteristics (Laney 2001, 1) Volume: Large quantities of data; 2) Variety: Data sources and the data itself differ; 3) Velocity: Data are added and updated continuously.

Big data could be used to gain more insight into specific PLN processes: to predict, for example, achievement and develop measures to improve teacher learning in PLNs. Developments in the field of technology, in particular, those occurring during the (post-) Covid-era, have led to a situation in which more and more data have become available and this is growing exponentially (e.g. social media, online learning environments, MOOCs) (Piety 2013; Williamson 2016). All these instruments register data about their users. These data could provide insight into user preferences, patterns of use and ways of learning in PLNs. Possibilities (and challenges) in the field of artificial intelligence (AI) are constantly arising. Systems exist that can discover patterns in large amounts of data, and these systems can replicate human reasoning, such as arguing, generalising, interpreting and learning from the past (Castelvecchi 2016). These systems can assist us in executing certain tasks and can even take over certain activities. AI can be used for speech recognition, categorising of pictures, process language and can make adaptive decisions based on (real time) data from sensors and other forms of digital data (Wang 2021). It could, perhaps, assist in studying the effectiveness of PLNs. These types of technologies, including data mining (e.g. text mining, audio mining and video mining) and data analysis (e.g. machine learning, model training and testing) (Fayyad, Piatetsky-shapiro, and Smyth 1996; Romero, Ventura, and De Bra 2004) could be used in PLN research.

Text mining, based on machine learning algorithms, aims at detecting patterns in texts that are related to a particular output (see, for example, Liu et al. 2018). In the context of educational science, in the study by Liu et al. (2018), reflective thinking of teachers was assessed in online learning environments using text mining. In determining the role of reflective professional inquiry (RPI), for example, PLN meetings could be (audio) recorded and (automatically) transcribed to document the extent to which RPI actually took place, rather than relying on self-report data only. A machine learning approach could be used in a pilot study, involving training a model to analyse and identify aspects of reflective and non-reflective dialogue. Once trained, the model should take just minutes to accurately and objectively process hours of recorded conversation and produce a report to indicate the extent to which teachers in a group were engaging in these types of problem-solving behaviours. Additional data, such as meeting notes and logs, or email communication between members about their PLN process, could also be used. Text mining, or even audio mining (Borgdorf, Veldkamp, and Poortman 2018), and machine learning could be used to make in-depth research into the PLN enactment process, for example, more efficient, valid and reliable. Social media (e.g. teacher Twitter posts or email messages, see Heldens 2017) and digital professional learning networks (Gruzd, Paulin, and Haythornthwaite 2016; Roessler and Westfall-Greiter 2018) also enhance studying actual enactment processes and teacher learning, through the analysis of the data and log files on these platforms, rather than solely relying on (retrospective) self-report data.

It is evident that digital education has undergone a considerable development as a result of the Covid-19 pandemic, demanding an even more fluent-data (Bayne and Gallagher 2021) approach in engaging both creatively and responsibly with learning data, available from online-learning environments. However, although the use of a big data approach in studying PLN seems promising, there are also several challenges to note here (Veldkamp et al. 2017), such as legal challenges, ethical and social challenges, and technological challenges. In conclusion, it is clear that more research is needed on whether and how big data can be used to study PLNs and to solve some of the methodological challenges raised in this paper.

Conclusions

In this theoretical discussion paper, we aimed to define further the concept of PLNs and to discuss challenges and recommendations in studying PLN impact to advance the field in showing more univocal impact on student outcomes. We proposed, as presented in Figure 1, that PLNs are defined as a group of educators (e.g. teachers, school leaders, possibly in collaboration with researchers, and/or policy-makers) coming together with others outside of their everyday community of practice with the intention of engaging in collaborative learning to improve outcomes for students. Student learning outcomes do not necessarily have to equate to achievement outcomes. Networks might also focus on issues of student wellbeing or equity or on engaging in a critical examination of the purpose and the aims of the curriculum (Brown 2020; Datnow and Park 2018). Furthermore, alongside this definition, we argue that there are both PLN enactment processes (including, for example, collaboration and PLN leadership) and influencing factors (operating at the individual, school context and policy level) at play. This conceptual model is visualised in Figure 1.

Apart from conceptualisation, the main challenges in being able to link PLN participation to outcomes for students concern research approach, research design and measurement. We first recommend the theory of action of PLNs in this study and the components need to be well defined, and this should be used to guide the evaluation. Secondly, PLN studies need to report more completely on background and objectives, methods (including participants, intervention, measures), results, and discussion (including limitations, generalisability and interpretation). Third, in terms of measurement, researchers need to conceptualise PLN elements further before they are studied, based on, for example, a systematic literature review.

Moreover, it is important to collect data in combination with methods other than only (retrospective) self-reporting; for example, using meeting (audio) observations and social media data. Data on outcomes for students related to PLN goals are indispensable in making the link. The measurement component should not be considered separately from the research design component. A big data approach, including audio and text mining, might help in making the analysis of patterns in and relations between these types of data more efficient and reliable. Given the software and hardware now available, we suggest that this provides a promising avenue for PLN research moving forward. Further research needs to shine more light on both the opportunities and new challenges related to a big data approach.

In summary, there is a need to define and study the processes and effects of PLNs more efficiently and effectively, in order to support PLNs in fulfilling the promise of increased teacher learning, improved outcomes for students, and ultimately sustainable school improvement at scale. We offer our discussion and proposed conceptual model as a contribution to the methodological development of this important field.

Acknowledgments

We would like to acknowledge our colleagues participating in the 2019 AERA research conference, organized by Alan Daly, Joelle Rodway, Chris Brown and Cindy Poortman; ‘Professional Learning Networks In Education: Tearing down boundaries for school and system improvement’, and especially our discussant, Dr Elizabeth Farley-Ripple, for their very helpful comments on an earlier version of this paper. This research conference was supported by a grant from the Education Research Conferences Program of the American Educational Research Association.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. The authors wish to thank Marieke van Geel, University of Twente, for her help with the visualisation of the theory of action.