A pragmatic meta-model to navigate complexity in teachers’ professional Learning

KEYWORDS:

• Professional Learning
• Meta-Model Framework
• Complexity
• Planning and Evaluation
• Pragmatism

Introduction

The voluminous literature on teacher professional learning and development presents varied definitions and perspectives on what constitutes professional learning and development. Several terms have been used in the literature in recent decades to describe teachers’ learning, most notably, in-service, continuing professional development, professional development, and more recently professional learning. These are often used interchangeably without critique. For the purpose of this article, we are intentionally using the term professional learning (PL) to reflect the complexity of teacher learning and teacher agency in the process of learning (Clarke and Hollingsworth 2002). Complexity ought to be considered in the design, enactment and evaluation of PL to allow stakeholders at the macro, meso and micro levels of the system to understand how complex systems operate, and impact PL (Opfer and Pedder 2011a). While teachers may receive ‘in-service’ or engage in professional development experiences, programmes, or activities, this may or may not lead to teacher PL and/or growth (King 2014, Boylan et al. 2018, King et al. 2022). PL is therefore an outcome of engagement in professional development (Liou and Canrinus 2020, McChesney 2022) and not something that is ‘done’ to teachers (Timperley 2007, p. 33). Rather, teachers are considered active participants using their agency to direct and enhance their own professional learning (Labone and Long 2016). This article intentionally adopts a broad meaning of the term ‘teacher’ to include teachers, leaders, paraprofessionals and others operating in the teaching and learning process within education systems. Noteworthy is that PL may be formal (e.g. arising from engagement with PD courses or workshops) or informal (e.g. arising from social interactions with others in school) (Spillane et al. 2011). PL may be individual or collaborative, transmissive or transformative (Kennedy 2014), but one of the key tenets of PL is critical reflection (Clarke and Hollingsworth 2002, Liou and Canrinus 2020) to make sense of new learning and how it relates to current practice and thinking. This brings to mind an oft-cited paraphrase of Dewey’s (1933) thinking: we do not learn from experience; we learn from reflecting on experience. Teacher learning-practice is not linear (Strom and Viesca 2020), it is a complex system influenced by various interactions with and between teachers, the professional development experience, and their contexts (Opfer and Pedder 2011a). Complexity theory allows for a systems-level view of how teacher learning can result in changes in practice and teacher growth (Strom and Viesca 2020). Complexity also allows for an exploration of how systems can support or hinder learning and practices (Opfer and Pedder 2011a). This article aims to shed light on this complexity.

Evaluation of the impact of PD activities or experiences has been cited as the weakest link in professional learning (Earley and Porritt 2014) with much focus to date on teacher satisfaction only. However, governments have sought more robust evaluation to examine the impact of professional development on teachers’ learning and ultimately student outcomes, arguably to justify continued investment (Boylan et al. 2018). However, because teacher learning is not a simple, linear process, it can be challenging to establish cause and effect between engagement with professional development experiences, activities or workshops, and teacher and student outcomes (King 2014, Rawdon et al. 2020). Instead, PL evaluation requires models to be cognisant of the complex, situated, and contextual nature of teacher PL (Boylan et al. 2018, Rawdon et al. 2020). Of note here is the significance of using the knowledge and understanding around core design features of effective professional development, learning processes, and other influences as outlined above to plan and evaluate PL – as planning and evaluating PL can influence implementation (Boylan 2021) and improve teacher and student outcomes (Snyder et al. 2012, King 2014, Schachter 2015, Philipsen et al. 2019). Evaluation frameworks need to be cognisant of the highly complex network of influences within schools and within and between systems that impact teacher learning-practice (Strom et al. 2021). Critiques of existing models highlight that no one model can ever be universally applicable for designers, researchers and evaluators at varying levels of the system (Boylan et al. 2018). Consideration of the purpose of teacher PL and allowing for the complexity of teacher learning-practice (Strom and Viesca 2020) requires a flexible and practical approach to planning and evaluation of PL. Such an approach is necessary to inform any action towards the planning or evaluation of PL in light of the increasing awareness of PL as a complex phenomenon. This paper proffers a conceptual meta-model of PL grounded in pragmatism to inform action in the design and evaluation of PL. It will firstly explore the aim and implications of the meta-model followed by outlining the methodology underpinning the development of the meta-model. Next, it will detail the constructs of the meta-model before outlining how the meta-model can be used at system, school and teacher levels.

The aim and implications of our conceptual meta-model

The paper builds upon two of the authors existing frameworks for designing (King 2014, Poekert et al. 2020) and evaluating PL (King 2014), whilst also drawing from the key constructs, theories, and processes of PL in the wider literature. This paper puts forth a conceptual meta-model that embraces complexity and roots itself in Deweyan pragmatism. We argue that adopting pragmatism answers Boylan’s (2021) calls to ground professional learning models in epistemological, ontological, and ethical commitments and suggests an approach to the design, enactment, and evaluation of professional learning grounded in inquiry. Dewey defined inquiry as ‘the controlled or directed transformation of an indeterminate situation into one that is so determinate in its constituent distinctions and relations as to convert the elements of the original situation into a unified whole’ (Dewey 1938, p. 108). This definition surfaces three essential points about pragmatic inquiry.

First, inquiry begins with an indeterminate and complex situation, which is acknowledged as problematic and in need of correction. As Boylan (2021) notes, the constructs that are accounted for in professional learning models, everything from antecedents to teacher reactions to teacher knowledge to teacher practice to school culture to sociopolitical context, are highly entangled and indeterminate. It is exceedingly difficult to tease out where one construct ends and another begins because they exist in relation to one another. In this way, we address ontological commitments by positing complexity as the reality or existence of the phenomena we aim to understand.

Second, Dewey’s definition of inquiry above acknowledges that engaging in inquiry is not merely a mental operation. Rather, the process of inquiry actually transforms the situation under investigation in an effort to resolve it. This is similar again to Boylan (2021) drawing parallels with quantum physics where the act of observing phenomena actually changes what we observe because attending to one feature of the phenomena can cause us to miss another. Similar dynamics are at play in the observation and evaluation of constructs involved in professional learning, for example, inquiring into teacher beliefs may cause those beliefs to change within the inquiry process. In highlighting this, we address the epistemological commitments by asserting that the constructs can only come to be known through inquiry while acknowledging that the act of inquiring into phenomena actually transforms the phenomena themselves.

Finally, Dewey’s definition of inquiry also helps to address Boylan’s (2021) call for clear ethical commitments. What separates inquiry from trial and error is that inquiry is controlled or directed through reflection. Biesta and Burbules (2003) point out that inquiry thus consists of the cooperation of two kinds of operations: existential operations, which actually transform the situation (as described above), and conceptual operations, the reflection or thinking. Thus, inquiry requires not only cognition but also directed action. If a problematic situation under scrutiny were simply reflected upon, without action being taken, the problem would remain indeterminate and no closer to resolution. If, on the other hand, action were taken without reflection, that trial and error would add nothing to understanding and, even if successful, would not lead to more intelligent action in the future. Inquiry thus implies a direction, and this is where ethical commitment comes to the fore. Pragmatism points in a particular ethical direction when engaging in inquiry to resolve problematic situations towards achieving an equitable, participatory democracy. Garrison (1994) explains,

Democracy was for Dewey the most logical form of government; it was less about voting than about equal participation by all in the conversation of humankind. Initiation into this conversation is the purpose of education, and it is the purpose of educational research to provide tools that aid this task. (p. 13)

A pragmatic approach stipulates not only that educational research should take aim at achieving democracy, but it should also do so using methods that promote the participation of the full citizenry, including members of various ethnic groups, races, and social class backgrounds. Doing so helps to transform the problematic situation that pragmatic inquiry seeks to resolve.

The meta-model proposed in this paper keeps in this tradition of understanding inquiry as a transformative process in both the design, enactment and evaluation of professional learning, seeking not only to understand a problem under scrutiny but also to amend the situation towards an equitable, participatory democracy.

Methodology

In an effort to develop a meta-model that could be utilised across various sectors of and stakeholders within education, we conducted a narrative review process which was strengthened by a systematised search (Ridley 2008) to identify literature across educational contexts that centred planning and/or evaluating PL across educational contexts. This approach allowed flexibility in selecting articles relevant to the multiple constructs involved in planning/evaluating PL (Paré et al. 2015). Our goal was to capture suggestions and critiques of existing models or frameworks used for planning/evaluation of PL that surfaced in literature, in an effort to develop a conceptual model that may also serve as a practical tool for academics and practitioners alike.

In line with a meta-design approach which allows for considering complexity, we began with a comparison of the two models that we had published independently, each of which was based on a review of the professional learning literature. The meta-design approach allowed for a comparison of the two models to identify what constructs were common and distinct between them, thereby ”mixing theories, concepts and processes” to build upon previous work as in meta-design approaches (Golson and Glover 2009, p. 2). Then, we sought to inform the development of the meta-model that would synthesise the two models while drawing from the broader literature, inclusive of seminal articles that had been published in the past as well as more recent literature that had been written since the publication of each of our own models.

We developed inclusion and exclusion criteria to produce quality articles for our proposed meta-model. These criteria included articles that focused on models or frameworks that incorporated professional learning or professional development in the contexts of schools or specifically working with teachers. We excluded articles beyond the years of 2014 through 2023 with any exceptions outside this range that were papers considered seminal to PL in education (e.g. Kirkpatrick, 1959; Stake 1967, Guskey 2002, Desimone 2009, Bubb and Earley 2010).

The first step in the search involved an exploration of the publications of the Professional Development in Education journal and their references, given the journal’s recent embrace of a complex turn in PL research. The search terms included ‘evaluation framework’, ‘evaluation model’, ‘impact’, ‘planning professional development’ which yielded nine articles of relevance to this research. A further six articles from the recent King et al. (2022) systematic review of reviews related to PL were included due to their explicit focus on planning and/or evaluating professional learning.

Additionally, we utilised Google Scholar, EBSCOHost, and Proquest, three large database search engines, to identify scholarly articles of interest. A first round search for ‘professional development’ in combination with other search terms including ‘framework,’ ‘model,’ ‘teachers,’ and/or ‘schools’ yielded eight relevant articles. A second round search for ‘professional learning’ in combination with the same search terms yielded an additional six articles. In total, 29 articles were collected to create our meta-model; however, following the removal of duplicates and conducting an additional synthesis of relevant literature, 24 articles were incorporated into the final analysis.

From the articles, we extracted the models and analysed the constructs inductively, comparing across models to identify common constructs and theories about the interaction and influence among the constructs. The articles were critiqued against and integrated with the broader PL literature, along with the authors’ existing frameworks (King 2014) and Poekert et al. (2020). The analysis highlighted various conceptualisations of learning processes that illustrated the complexity at work among the constructs, contexts, and stakeholders, and these were synthesised into the meta-model presented in this article which has synthesised the constructs into three major constructs in the meta-model; Context, Experience, and Outcomes. The resulting meta-model draws upon the scholarly literature to yield a set of constructs and sub-constructs, diagrammed below, along with examples of how those constructs can be employed in the enactment and evaluation of PL, including both formal activities and informal interactions.

The conceptual meta-model

The meta-model aims to serve as a diagram, as defined by Deleuze (1995, p. 44): ‘a map, or rather a series of superimposed maps.’ Such diagrams model the rhizomatic nature of teacher learning. Because the model is rooted in pragmatism, it affords flexibility in its application based on the goal(s) that a user is trying to achieve. Previous taxonomies of professional learning (e.g. Kennedy 2014) posit that these goals might range from the transmission of discrete skills or knowledge to the development of competencies to the transformation of professional identities. It stands to reason that as the goals of professional learning vary, so will the constructs that must be accounted for in the design and evaluation of PL. It also stands to reason that the sequential connections and interactional directions from one construct to another might appear different within distinct PL efforts that might operate at different macro-, meso-, and micro levels. What follows here is a description of the major constructs and sub-constructs included in our meta-model, introduced one at a time accompanied by diagrams that layer the constructs upon one another. This description will be followed by illustrations of how these constructs can be flexibly operationalised in the enactment and evaluation of PL according to the goals of the activity and the level (system, school, and teacher) at which the activity occurs. The three major constructs included in our meta-model are context, experience, and outcomes (CEO). We have subsumed all other constructs under these three headings.

Context

The construct of context in this meta-model aims to be flexible enough to allow for application in the complex system of teacher learning-practice and research. Context accounts for the circumstances that both precede and surround the PL at the macro, meso, and micro levels, ranging from the antecedent factors and previous experiences of the participants to the organisational culture of the school and school system to the larger community (see Figure 1). Understanding the contexts within which teachers are operating is pivotal to supporting effective teacher PL (Merchie et al. 2016, Sancar et al. 2021, McChesney 2022). The macro context is highlighted by Hallinger (2018) as he points to the importance of the wider economic, socio-cultural, political, and historical contexts influencing leadership practice and teacher PL. The principal and teachers need to understand the schools’ context and culture related to teacher PL that may have been helped or hindered over time by these broader contexts. This macro-level context is also acknowledged as being entangled with teacher PL by other key researchers in the field, for example, Clarke and Hollingsworth (2002); Boylan et al. (2018); Poekert et al. (2020); Boylan (2021); Strom et al. (2021) and therefore warrants inclusion in this meta-model.

Figure 1. Context construct in the professional learning meta-model.

At the meso level, school contexts are pivotal in supporting teachers’ learning via leadership support at the school, institutional, and community levels (Campbell and Osmond-Johnson 2018, Poekert et al. 2020). At the school level, there are complex systems and challenges, e.g. cultures, that the leader may need to address. Leaders may need support to be enabled to deal with these challenges and to foster professional agency, create organisational capacity for change, and to empower teachers to develop collaborative learning cultures (King 2011, 2016) such as communities of practice. Providing time and funding for engagement with professional development experiences, activities, or workshops followed by enactment and reflection has been increasingly highlighted as central to sustained engagement with PL (McChesney and Aldridge 2019, Liou and Canrinus 2020, Rawdon et al., 2020; Tonga et al. 2022, King et al. 2022). Additionally, an advocate or change-agent in the form of teacher leadership to support PL in schools is advocated (Poekert, 2012; King 2014, 2016, King and Holland 2022). The institutional-level context which refers to the education system within which a school is nested, a national, regional, and/or district system, also influences leadership practice. System centralisation or decentralisation along with management and accountability systems influences leadership practice (Hallinger 2018). Equally, the local community context within which the school is situated, including urban or rural, and socioeconomic status, may have a significant influence on the leadership of professional learning in the school (Hallinger 2018), thereby warranting the inclusion of the various contexts in the meta-model. Adopting a complexity stance requires an awareness of the multi-directional relationship and impact that exists both within the meso level and between the meso and the micro and macro levels.

At the micro-level context is the individual teacher-related factors which King (2014) and Poekert et al. (2020) both highlighted in their respective frameworks under antecedents and/or baseline. The significance of the individual teacher’s characteristics (Merchie et al. 2016, Sancar et al. 2021, McChesney 2022, Prenger et al. 2022), epistemologies (Sumsion et al. 2015), including teacher motivation to engage with PL, their expectations, along with their current knowledge, skills, attitudes, beliefs, experiences, and identity are highly influential in planning for teacher engagement with and the outcomes of PL (King 2014, Lee et al. 2021, Audisio et al. 2022). Boylan et al. (2018, p. 134) called for further development of models to include a focus on ‘the life course and identity of teachers’ as he later highlights that teachers’ attitudes, values, and beliefs are ‘entangled’ with engagement with professional development experiences (Boylan 2021). Developing a positive teacher-learner identity (e.g. as a writer or mathematician) has also been highlighted by Kennedy et al. (2022) as a means of supporting changes in beliefs and practices.

Similarly, having evidence of a teacher’s baseline knowledge and skills is essential for setting goals (Bubb and Earley 2010, Earley and Porritt 2014) and planning PL within the teacher’s zone of proximal development (Vygotsky 1978) in a bid to maximise impact. This reflects the importance of ‘teacher voice data’ when planning and evaluating professional development experiences (McChesney 2022). Therefore, consideration of context at all three levels is a central construct when planning and evaluating PL. Given the importance of planning PL at the outset for maximum impact (Bubb and Earley 2010, King 2014), Table 1 indicates some key questions to support this process.

Table 1. Planning and evaluating PL: Context.

Context	Possible questions to be explored	Processes for planning/evaluating	Example
Macro (Economic, socio-cultural, political, historical contexts including central government	Is the focus of the PD experience aligned with national and international policies? What political, cultural, historical factors, and material conditions and power flows may influence teacher engagement with PL? For example: Has teacher PD historically been seen as something that is ‘done’ to teachers? Is the focus on the enactment of equity practices or preserving the status quo? Is accountability the key driver or enhancing teacher quality? Is teacher learning-practice conceived as a linear or complex process? Is teacher learning-practice conceived as an individual or networked process? Are teachers empowered to enact agency? Who is funding and/or evaluating this PL? How is it being evaluated?	National and international policies	Opfer and Pedder 2011a and b)Strom and Viesca (2020)Boylan et al. (2018)Hallinger, 2018Rawdon et al. (2020)
Meso (School, institutional and community levels)	As a school where are we now (in relation to our goals)? What do we want to achieve? Is engagement with the PD voluntary or mandatory? Is there alignment between district/school leadership values and teacher values? What supports do teachers//leaders/facilitators need to create organisational capacity for change e.g. understanding of complex systems, time, funding, resources, advocate/teacher leadership? What products/processes will support teacher learning-practice e.g. policies … ? Are teachers being empowered to create collaborative cultures to allow for reflection and enactment cycles? Will there be continued support through the reflection and enactment cycles to support enhanced learning/pedagogical content knowledge (PCK)?	District/Regional and School policies District/School meetings School Records Focus Groups Interviews Questionnaires/Surveys on school culture Teacher leader competencies survey	King 2014 ,King (2016) Opfer and Pedder 2011a ()Poekert et al. (2020)Audisio et al. (2022)Merchie et al. (2016)
ro (Teacher Antecedents, including baseline knowledge and skills)	Does the PD align with individual teacher interests, needs and/or identity? What is the personal motivation for engaging in this PD? Why are you engaging with this PD experience or activity? What are your expectations? What is your current level of knowledge and skills related to … ? What are your current beliefs and values in relation to … ? What do you want to achieve (goals/targets)? What do you want the students to be able to do (cognitive, affective and/or psychomotor levels)? Evidence based e.g. students work, behaviour, skills … What are your concerns about implementation? Who is evaluating the PD and how?	Interviews, focus groups, questionnaires e.g. Survey using open ended questions, rank order or rating scales to be used for planning and evaluating PD experiences. Informal practices e.g. standing on a line or placing a peg on a line to represent your current level of knowledge, skills or efficacy e.g. lacking confidence - highly confident.	Clarke and Hollingsworth (2002) King 2014 Sumsion et al. (2015) Poekert et al. (2020) Lee etal., 2021 Audisio et al. (2022)

Experience

Experience accounts for the constructs involved in PD, including core design features of the PD experience as well as the learning processes and theories underpinning their design, such as the level of collaboration and the use of situated, inquiry-based learning techniques to support teacher PL. Equally important for both planning and evaluating teacher PL is the construct of the professional development experience (see Figure 2). To date, much evaluation at the macro, meso, and micro levels has focused on teachers’ reactions and satisfaction with the professional development experience. For example, King’s (2014) model focused on teacher satisfaction with the overall experience, the model, content, venue, facilitators, and so forth. Arguably teachers’ reactions and satisfaction are influenced by the design factors of the professional development experience along with other teacher-related factors.

Figure 2. Experience and context constructs in the professional learning meta-model.

Design factors include the core features of effective professional development when planning PD experiences to support teachers’ PL; content focus, active learning, collective participation, coherence, and duration (Desimone 2009), external providers/specialists, and leadership for PL (Cordingley 2015), coaching and expert support to allow for feedback and critical reflection, as well as connecting theory, practice, skills, and knowledge (Darling-Hammond et al. 2017). Given this meta-model is rooted in the complexity of teacher learning-practice, it allows for practical flexibility in its application of which of these core design features will work for the particular goal of the professional development experiences teachers are engaging with, in their context, at any given time.

When planning teachers’ PD experiences there is also a need to consider PD processes as being situated (Labone and Long 2016, Boylan et al. 2018, Liou and Canrinus 2020) and job-embedded (Powell and Bodur 2019) to allow for ‘enactment and reflection’ learning processes (Clarke and Hollingsworth 2002), reflecting the need to draw upon situated and experiential learning theories to support teacher change and teacher learning-practice. Within complexity it is important to acknowledge the challenges with implementation or enactment of PL and make space for problem identification and solutions to support the application of PL (King and Holland 2022).

Teacher agency and autonomy are also key processes to be included when planning teacher PD experiences and PL (Clarke and Hollingsworth 2002, Opfer and Pedder 2011a and b, Evans 2014, King 2014, 2016, Liou and Canrinus 2020). Including teachers in the design of the professional development experience can maximise teacher learning-practice (Saderholm et al. 2017, Holland 2021). Underpinning agency and autonomy are a number of theories including self-determination theory (learner-driven) (Ryan and Deci 2006), progressive education theory (learner-centred) (Dewey 1916), and hope theory (goal-focused) (Snyder et al. 2003). Adopting a complexity stance would allow learners, facilitators, designers, and researchers at all levels of the system to explore a wider range of theories to impact on the design, enactment and evaluation of PL.

This meta-model for planning and evaluating PL also includes a focus on PL as something that is situated, collaborative, and has a social dimension to reflect Boylan’s et al. (2018) call for further development of models in this area. Underpinning the collaborative and social dimension is social constructivist theory (Clarke and Hollingsworth 2002, Opfer and Pedder, 2011a and b) to support transformative learning (Kennedy 2014) through collaborative inquiry models of PL such as communities of practice (Lave and Wenger 1991).

As with core design features, there is not a checklist of learning theories applicable to all situations. Depending on the goal of the PD experience, different design features, learning theories, and processes will be drawn upon to support teacher learning-practice as teacher-learning practice is complex. Table 2 outlines some key questions to support those planning PD experiences to support teachers’ PL.

Table 2. Planning and evaluating PL: PD experience.

Planning PD experiences
PD experience	Possible questions to be explored	Data gathering methods	Example
Design features	What PD activities, experiences and/or model do teachers need to gain the required growth or knowledge, skills, attitudes etc? Does the model of professional development match the intended goal? Coherence: Are the goals predetermined or negotiated with teachers? Are the goals aligned with external factors e.g. standards, policies … ? Is the content aligned with teachers’ or school needs and/or system needs? Is there a focus on ‘how’ to teach (pedagogy and PCK)? Is it research/evidence-based in terms of impact on students? What knowledge and skills will be needed to facilitate teacher learning-practice? Is the PD within the teacher’s Zone of Proximal Development? Is the PD design structured, feasible, focused and evolvable? Does the experience allow for active learning? Does the experience allow for collective participation? Is there a focus on individual and collective responsibility for student learning? Is the experience for a sufficient duration to support teacher learning-practice? Is there a need for an external provider or facilitator or is the expertise available in-house?Is the type of facilitation evolvable? Is leadership for PL planned to support teacher learning-practice? Is there a need for coaching or mentoring to support teachers or PL facilitators to support teacher learning-practice, reflection, and enactment?	Checklists or rubrics Interviews Fidelity of implementation tables Questionnaires Observations (video/audio)	Desimone (2009) Cordingley (2015) Darling-Hammond et al. (2017) McChesney (2022) Schachter (2015) Clarke and Hollingsworth (2002) Merchie et al. (2016)
Learning processes and theories	Which learning theories and processes will support the intended goal e.g. social constructivist, situated, self-determination theory Does the professional development experience support situated and collaborative learning? Does the professional development experience support problem identification and solutions for enactment? Does the professional development experience support teacher agency and autonomy? Which learning theories might support teacher learning-practice e.g. situated and experiential learning, social constructivist theory, self-determination theory, complexity theory, hope theory, empowerment theory etc?	Checklists or rubrics Interviews Fidelity of implementation tables Questionnaires Observations (video/audio)	King and Holland (2022) Liou and Canrinus, 2020 Snyder et al. (2003) Dewey (1916) Ryan and Deci (2006) Holland (2021)

Outcomes

Outcomes account for the outputs and outcomes of engagement in PL, ranging from the immediate reaction of educators to the experience to the long-term impacts on students’ quality of life outcomes. Not only must we account for outcomes temporally, but we must also account for individual and collective short and long-term outcomes on students, teachers, schools, and the larger system and community they compose (see Figure 3). The outcomes among these stakeholders influence the context in which future PL might occur.

Figure 3. Professional learning meta-model with context, experience, and outcomes (CEO) constructs.

Student learning outcomes have always been central to the purpose of PL. Guskey’s (2000) inclusion of student learning outcomes can be connected back to Kirkpatrick’s (1959) notion of organisation results, in that the results that schools aim to improve connect to student learning. The models outlined in both King (2014) and Poekert et al. (2020) also highlight the importance of student learning outcomes. Additional research (Bubb and Earley 2010, Earley and Porritt 2014, Merchie et al. 2016, Baird and Clark 2018, McChesney 2022) has explored the impacts of PL on student learning ranging from the cognitive (e.g. performance and attainment) to the affective (e.g. attitudes and dispositions) to the psychomotor (e.g. skills and behaviours) as outlined by Bloom (1956). The Poekert et al. (2020) model also separates out immediate impacts on student achievement assessments from longer-term impacts on student quality of life.

There are also a number of teacher outcomes that have been documented in the literature, which are typically portrayed as intermediary outcomes in a conceptual chain of constructs connecting PL experiences to the intention to improve student learning outcomes. The first construct that is frequently reported is participants’ immediate reaction to the professional learning experience (e.g. Kirkpatrick, 1959; Guskey 2000), presumably because data on reactions are more easily acquired through participant surveys than some other outcomes of interest that are more difficult to measure.

Beyond initial reactions, the PL literature documents the impact on two additional broad categories of teacher outcomes: teacher learning and teacher practice. Teacher learning includes new knowledge and skill as well as changes in beliefs, attitudes, and dispositions (King 2014). Teacher practice includes changes, with attention to the degree and quality of change, in the behaviour of individuals, both personally and professionally, as well as the collaboration of groups of teachers and the overall culture of a school (Kirkpatrick, 1959; Guskey 2000, Holton 2005, Bubb and Earley 2010, Earley and Porritt 2014, King 2014, Merchie et al. 2016, Poekert et al. 2020).

Dating to at least Kirkpatrick’s (1959) model for the evaluation of training programmes, researchers have attempted to parse out the constructs that influence the relationship among professional learning experiences and the outcomes they are intended to produce in schools and organisations. The disaggregation of the constructs involved in this relationship has increased over decades of research and is intended to better illustrate the nuances at work. For example, researchers from Guskey (2002) to Desimone (2009) have posited different sequences of interaction among the constructs. Guskey’s sequence hypothesised that professional development leads to changes in teachers’ classroom practice, which lead to changes in student learning outcomes, which lead to changes in teacher beliefs and attitudes. Desimone proposes a different sequence in which professional development leads to increased teachers’ knowledge, skills, beliefs, and attitudes, which leads to change in instruction, which leads to improved student learning. Both sequences are linear and arguably plausible within the individual lived experience of a teacher’s professional learning. However, more recent conceptualisations of PL reveal a more reciprocal interaction between the constructs, influenced by the teacher, their context, and the PD experience (past and present) or activity (Opfer and Pedder 2011a , Liou and Canrinus 2020). Therefore, the conceptual model we use to illustrate the relationship among the constructs accounts for this complexity by allowing flexibility in the sequence. Table 3 below outlines some key questions to support those planning and evaluating PD experiences for particular outcomes.

Table 3. Planning and evaluating PL: Outcomes.

Outcomes	Possible questions to be explored	Data gathering methods	Example
Student Short-term Long-term	Affective: What were the students’ attitudes and dispositions in relation to the new practices? Cognitive: How are student performance and attainment following the new practices? Psychomotor: What new skills and behaviours did students develop or enhance following the new practices?	Interviews, focus groups with students, questionnaires, formal assessments, pre- and post-intervention testing, Field-notes Portfolios Self-reports and/or Observations	Bloom’s Revised Taxonomy: Cognitive, Affective, and Psychomotor Bloom’s Taxonomy | Centre for Teaching Excellence | University of Waterloo
Teacher Short-term Reactions Learning Practice Long-term Personal/professional identity	Are teachers generally satisfied with the overall PD experience, content, duration, opportunity for dialogue, venue, facilitators, time for reflection … ? What knowledge, skills, attitudes, and beliefs were acquired, enhanced, or affirmed? Were expectations and goals met? How would you describe teachers’ beliefs, attitudes, and efficacy toward new knowledge, skills, and/or practices? How would you describe the quality of use and understanding of new and improved knowledge and skills: non-use, orientation, preparation, technical, accepted, critical, or discontinued? Did the PD experience allow for problem identification and solution? Did the learning and/or practice diffuse to other adults/students in the school or in other schools?	Quantitative and qualitative methods e.g. pre-, during, and post-intervention testing, interviews, focus groups, observation, questionnaires, or a combination of these. Verbal feedback video recordings Written reflections Videos of practice	King 2014 Poekert et al. (2020) Hall and Hord (1987) Ahadi et al. (2021)
School Collective Organisational	Did the PD experience support situated and collaborative experiential learning, agency and teacher autonomy? What forms of collaborative practice followed engagement with the PD experience e.g. development of PLCs, CoPs, peer observation … ? Is there evidence of collective responsibility for student outcomes? What, if any, new or improved processes were devised following the PD experience e.g. a new approach to needs analysis … ? What products, if any, arose from engagement in the PD e.g. an improved or new policy … ?	Documentary analysis, focus groups, questionnaires, or a mixture of these	King 2014 Poekert et al. (2020) Merchie et al. (2016) Ahadi et al. (2021)
System/Community	How has the PD experience influenced the operations and culture of the governance system in which a school operates? Did the PD experience lead to more engagement with or other impacts on the community? Did the PD experience lead to collaborations with other schools or Higher institutions? Did the PD experience lead to collaborations with other support services e.g. libraries, psychology services, speech and language services?	Quantitative and qualitative methods e.g. pre-, during, and post-intervention testing, interviews, focus groups, observation, questionnaires, or a combination of these.	King 2014 Poekert et al. (2020) Merchie et al. (2016)

Pathways

In this section, we demonstrate how learners, researchers, designers or evaluators can operationalise the meta-model within the context of planning and evaluating PL using the meta-model and its component constructs and subconstructs. Using examples at a system (see Figure 2), school (see Figure 3), and teacher (see Figure 4) level, we illustrate how PL stakeholders can use the meta-model to operationalise the CEO constructs of context, experience, and outcome. The PL efforts in each of the examples were prospectively designed and evaluated using existing PL frameworks at the time of their implementation. Here, we aim to retrospectively apply the meta-model to demonstrate how it can be broadly applied at each level. The examples draw upon previous work undertaken by the authors. Readers should note how only the subconstructs that are relevant to the planning or evaluation of the specific example of PL are displayed in the illustrations, while the others are removed. Also of note is how the numbers within the model illustrate the conceptual flow from one CEO construct to the next, typically beginning with the outcome that is to be achieved and then moving to other constructs, either context or experience depending upon how the PL is planned or evaluated.

Figure 4. Professional learning meta-model – system-level example.

System level

An example of planning and evaluating PL at a system level can be found in the work of Literacy Matrix described by Fairman et al. (2023) and evaluated by the Frank Porter Graham Child Development Institute (Frank Porter Graham Child Development Institute 2022). This large-scale professional development effort was launched across the state of Florida in 2018, and focused on improving elementary teachers’ knowledge of teaching practices grounded in the Science of Reading (Figure 4, No. 1) and student reading proficiency (Figure 4, No. 2). At the macro level, the effort was launched within the context of a significant statewide push (Figure 4, No. 3) by the Department of Education and the state’s flagship university. The effort also received the philanthropic support of a popular and influential author who contributed to the support it received from the state legislature, and at the meso level from local school districts across the state as it was rolled out (Figure 4, No. 4). At the micro level, the effort was piloted with 331 teachers who provided feedback on early prototypes of the video-based PD resources and the design of the PD experience and subsequently scaled to 2,073 teachers across Florida. Among participating teachers (Figure 4, No. 5), pre-test knowledge assessments and demographic surveys were also gathered to understand the antecedent circumstances that influenced the teacher’s experience and outcomes of the PL initiative.

The PD experience was designed to include several integrated components: instructional modules, resources (such as model lesson plans and videos modelling teaching practices), opportunities for teachers to submit artefacts for formative feedback (such as video-taped instruction and written reflections), video-taped coaching feedback for individual teachers, and options to share resources with colleagues through social media-style interactions (Fairman et al. 2023, p. 203) (Figure 4, No. 6). A series of video modules and written resources comprise the PL curriculum and aim to build teachers’ pedagogical content knowledge around phonological awareness, decoding, fluency, vocabulary, and comprehension. The learning process (Figure 4, No. 7) utilises inquiry-based, job-embedded enactment and reflection because

teachers work individually through the modules and assessments in a sequential manner for each of the domains of reading, moving on to assessment instruction, and intervention strategies. Each module ends with a practicum where teachers demonstrate their learning in action in their classrooms. (Fairman et al. 2023, p. 203)

The outcomes of the PL experience that were monitored within the effort centred on the impacts of the PL experience on teacher knowledge (Figure 4, No. 8), measured as the difference between pre- and post-test knowledge assessment around the Science of Reading. For example, just 2% of the participating teachers demonstrated mastery (defined as 80% or higher on the knowledge and practice assessments) at the pre-test, whereas 99% demonstrated mastery by post-test (FPGCDI 2022). Student cognitive outcomes (Figure 4, No. 9) were also evaluated and demonstrated significant impacts on the learning outcomes of students in participating classrooms when compared to students of non-participating teachers (FPGCDI 2022). Overall, results found significant growth in both teacher learning and short-term student learning outcomes across the entire system (Figure 4, No. 10). Additional data were gathered from teachers about their reactions (Figure 4, No. 11) to the PD resources and experience in terms of their relevance to the classroom as well as teachers’ enjoyment and engagement to inform revisions to the content and future development, and 77% of participating educators reported positive feedback (FPGCDI 2022).

School level

An example of planning and evaluating PL at the school level was undertaken by Brennan, King and Travers (2019) in the context of one elementary school in the Republic of Ireland where the goal was to support ten teachers (eight classroom teachers, principal, and deputy principal) to develop inclusive pedagogical practices to meet the needs of students with special educational needs (Figure 5, No. 1). Ensuring goals align with policy and practice at the macro and meso levels is important to secure support. For example, at the macro-level inclusion is high on the international and national agenda, and support for this focus was therefore forthcoming at the meso level of the Professional Development Service for Teachers who supported teachers and schools (Figure 5, No. 2 and 3). The facilitator of this PD experience worked for this service and was undertaking doctoral studies at the time. She had previously supported teachers in the school in the area of literacy and they also looked for additional support in meeting the needs of all learners. Keeping the core design features of PD experience in mind, the school principal was approached initially to ensure that leaders would create organisational capacity for change through providing time, funding, and resources for engagement with a professional learning community (PLC) within the school (Figure 5, No. 4). Asking leaders for support in empowering teachers involved in the PLC to create collaborative cultures for reflection and enactment was also part of the planning process to support situated and experiential learning processes (Figure 5, No. 5). Leaders were also asked to be part of the PLC to show their support and understanding for the same (King 2016). At the micro level (Figure 5, No. 6) teacher autonomy and agency were considered in terms of looking only for volunteers who were interested in developing inclusive pedagogy. Antecedents and baseline were considered at the first PLC meeting in terms of the teachers’ motivation, expectations, and goals and revisited again using individual interviews after the final PLC meeting (Figure 5, No. 6). A pre- and post-test efficacy for inclusive practices survey (TEIP) along with sentiments, attitudes and concerns about inclusive education scales (SCAIE-R) (Forlin et al. 2014) were administered to understand the antecedent circumstances that influenced the teacher’s experience and outcomes of the PL initiative.

Figure 5. Professional learning meta-model – school-level example.

Planning the PD experience was informed by social constructivist and situated learning theories aimed at supporting transformative learning (Figure 5, No. 7). The content was underpinned by the research and evidence-based approaches including the Inclusive Pedagogical Approach in Action framework that had been shown to positively impact on teachers enacting inclusive pedagogy (Florian and Spratt 2013). Several core design features from effective PD were adopted (Figure 5, No. 8) e.g. content focus (inclusion), active learning (reflection and enactment cycles), collective participation in the PLC (using pedagogies of critical dialogue and public sharing of work (undefined)), coherence with individual teacher needs and interests, duration (monthly workshops over a period of 6 months), along with external facilitation by an ‘expert’ in inclusive pedagogy. Individual teachers’ attitudes, beliefs, and values linked to inclusive pedagogy were explored as these are ‘entangled’ with PD experiences. Teachers largely determined the direction of the PLC in terms of focus and actions, e.g. they chose to focus on ‘differentiation by choice’ as an inclusive pedagogy. At each workshop, teachers engaged in the pedagogies of public sharing of work and critical dialogue on their practice and on student learning. Action plans for the following months were agreed and teachers critically reflected on their own learning from the workshop.

Outcome of engagement with the PLC was determined by ongoing teacher reflective learning logs, individual interviews following the final PLC, teaching observations in classrooms and the post-test TEIP survey and SACRE-R scales (Forlin et al. 2014). While changes were not statistically significant, they did demonstrate evidence of a small improvement in belief in and efficacy for inclusive practice. An observation schedule was designed to reflect the Inclusive Pedagogical Approach in Action framework and sub-constructs from King’s (2014) framework for evaluation; teacher reactions, learning and practice, collective learning, and overall impact at a cultural or organisational level (Figure 5, No. 9). For example, the collaborative culture arising from the PLC led to ongoing open dialogue and public sharing of work along with enhanced professional relationships beyond the PLC, but within the school, which led to enhanced communicative competencies for teacher collaboration. Short-term student outcomes were reported by teachers and through observations and ranged from affective to psychomotor and cognitive outcomes (Figure 5, No. 10).

Given the importance of the temporal nature of teacher PL a follow on study was carried out 2 years later (Brennan and King 2022) to explore the sustainability of inclusive pedagogy practices at an individual, collective, and wider school level. Interviews (n = 9) and observations (n = 5) revealed a collaborative culture arising from sustained enactment of teachers’ individual and collaborative practices of inclusive pedagogy for those originally involved, and for some additional staff (Figure 5, No. 11).

Teacher level

An example of planning and evaluating PL at the teacher level was undertaken by King and Holland (2022), using a meta-model, in the context of eight early career teachers from different elementary schools in the Republic of Ireland who came together as a community of practice (CoP) within a school–university partnership model (Figure 6, No. 1). The domain or goal within the community was leadership for inclusion (Figure 6, No. 2) which involves the transformation of professional identities of growth as a teacher, researcher, and leader all centred around personal growth (Poekert et al. 2016). All of the teachers had undertaken a major specialism in inclusive and special education as part of their undergraduate degree and wanted to stay committed to their moral purpose of leadership for inclusion. This is an example of teachers at the micro level context using their individual and collective agency to engage in PL that aligned with their personal interests (Figure 6, No. 3). A meta-model approach was used when planning this CoP using the above constructs of Context, Experience, and Outcome along with constructs around teacher leadership development (Poekert et al. 2016) and the six facets of equity for inclusion (Grudnoff et al. 2017) along with participatory action learning action research (PALAR) processes (Chevalier and Buckles 2013). Given the goal of the PD experience aligns with transformation of professional identities the three constructs of Context, Experience, and Outcome were used but to varying degrees. The context did establish a baseline and explore antecedents (Figure 6, No. 3) using activities during the first CoP meeting, e.g. recording individual goals and hopes on an e-technology platform (Trello).

Figure 6. Professional learning meta-model – teacher-level example.

Planning the PD experience was influenced by the key subconstructs of core design features and learning processes and theories. Core design features of content focus (leadership for inclusion, was set as a group goal and mission by the group members (PALAR process)), active learning (through use of target setting and action plans (PALAR process)), collective participation in the CoP, coherence with individual needs and interests, duration (eight workshops between November 2017 and April 2021), along with external facilitation through the school–university partnership (Figure 6, No. 4) were prospectively considered. Additionally, exploring and solving problems related to implementation, managing conflict and change (PALAR processes) were important processes to support teacher growth and teacher learning-practice. Underpinning the design was the prospective use of learning theories, including, for example, ‘progressive education’ theory (learner-centred) (Dewey 1916, 1933), ‘self-determination’ theory (learner-driven) (Ryan and Deci 2006), and hope theory (goal-focused; Snyder et al. 2003) (Figure 6, No. 5). Ongoing data collection about the PD experience was undertaken at and between each CoP workshop to ensure alignment with the above theories and design features, reflecting the importance of the inclusion of teachers in the design of PD activities (Saderholm et al. 2017). For example, protocols and norms were revisited/amended on Trello.

Within the construct of Outcome the key sub constructs explored related to the impact on teachers personally and professionally, individually and collectively as the goal related to long-term outcomes of transformation of teacher identifies to include identity as a leader, researcher, teacher all centred on personal identity (Figure 6, No. 6). Teachers were seen as the central beneficiaries of their own engagement with the PD experience (Boylan et al. 2018, Holland 2021). While the focus was ultimately on leadership for inclusion, the impact on student outcomes was not a direct focus in this PD experience or evaluation. Data gathering included artefact generation such as target setting and action plans uploaded to Trello, recorded CoP workshops, ‘Talking heads’ videos where teachers reflected on and shared the impact of the CoP on their identities.

Acknowledging the temporal nature of teacher learning this continues to be evident in their teacher leadership for inclusion via conference presentations, journal publication, sharing of and/or leading practices both within their individual schools and with teachers from other schools (Figure 6, No. 7). Noteworthy in this example is the lack of emphasis on the meso level of the school. This was a bottom-up approach at the micro level with support from a school–university partnership at the meso level. Further development of teacher leadership for inclusion within these schools requires a focus on the meso level to gain support for this work within the various schools.

Conclusion

This article intentionally recognises the difference between the terms ‘professional learning’ and ‘professional development’. It appreciates teacher learning as a non-linear, complex process that makes planning and evaluating PL somewhat challenging. While several models or frameworks of evaluation exist, this paper aims to answer more recent calls to show how such models are to be used. This article proposes a meta-model of professional learning that attempts to synthesise the constructs, theories, and processes accounted for in published models to date. It builds upon two previous well-known models (King 2014, Poekert et al. 2020) and a broader PL literature. It is hoped that the use of the terms professional learning and professional development throughout this article have added some clarity towards using the terms coherently in different situations.

This meta-model is cognisant of the complex, situated and contextual nature of teacher PL which operates within a network of influences within schools and systems that impact teacher PL and learning-practice. The meta-model is also reflective of how the constructs within the model are all entangled in various ways, e.g. between macro policy context and PL at the individual teacher level. PL design must seek to acknowledge and understand how they are entangled and the implications this has for the design. By grounding the meta-model in pragmatism to inform action in the design and evaluation of PL, we aim to address critiques that challenge scholars to highlight the epistemological, ontological, and ethical commitments that inform methodological decisions and approaches to PL.

The article shows how to operationalise the meta-model through empirical illustrations and processes at system, school and teacher levels. The goal of the PL will determine which CEO constructs and sub-constructs are relevant for consideration in planning and evaluating PL. Ultimately, this meta-model can guide all stakeholders to navigate the complexity within PL and promote language around the description of the CEO constructs that are both accessible across stakeholder groups and internationally relevant. It is hoped that the questions in the above tables may help stakeholders plan the various pathways as outlined in the above figures.

We argue that the model is also flexible enough to account for the complexity and nuance of individual experience as participants engage in professional learning while still highlighting the patterns that emerge across participants. This flexibility also affords discretion to both the PL designer in the selection of tools and approaches and the researcher in the selection of methods and instruments that are fit for purpose. It is our hope that this model will make both a theoretical and practical contribution by synthesising the conceptualisations of professional learning to date and serving as a platform for future theoretical and practical development.

Disclosure statement

No potential conflict of interest was reported by the author(s).