Quantifying professional learning

In his book The Bridge of San Luis Rey, Thornton Wilder ([1927] 2000) tells the fictional stories of five individuals who died when a rope bridge collapsed in Peru in 1714. The bridge had been used for over a century by travellers crossing a gorge on the road between Lima and Cuzco and was a seemingly indestructible structure, created by the highly reliable engineering skills of the Incan people many years earlier. The way in which Wilder narrates the personal histories of the five individuals makes the book a modern classic, but there is a key theme to the story which provokes theological and philosophical questions for one of the characters, Brother Juniper, a Franciscan missionary from northern Italy who witnesses the tragedy.

Brother Juniper asks why these five people were the ones on the bridge when it collapsed. Had they been particularly bad in their earlier lives to have been singled out for such a fate? In an attempt to answer this question scientifically, Brother Juniper researches the backgrounds of the five individuals by interviewing people who knew them closely. He busied himself for six years, ‘knocking at all the doors in Lima, asking thousands of questions, filling scores of notebooks’ (p. 11), eventually compiling an enormous book detailing the life histories of each of the victims. He was, in effect, compiling rich research data from which he thought he would be able to see patterns that would explain why the lives of these people had singled them out for death in the gorge. He then goes a step further. In attempting to find a more scientific basis for his ideas, he feels that he needs to quantify his findings. He needs to find ‘proof, tabulated proof’ of whether the fate of people is predetermined. When pestilence hits Puerto, his village, and a number of inhabitants die, he attempts a statistical classification of a sample of 15 victims and 15 survivors, rating each individual on a score of 10 as regards their Goodness, Piety and Usefulness to Society. He even uses negative scores for Goodness where an individual was ‘not merely bad …: he was a propagandist for badness’ (p. 113). He produces a table listing each of the inhabitants sampled and classifies each person using the scores for Goodness, Piety and Usefulness allotted to them. In its raw form, the extract from the table given in Wilder’s book is reminiscent of school league table lists (identifying which are the ‘good’ schools) or, on an individual basis, teacher performance scores.

In The Bridge of San Luis Rey, Brother Juniper’s book is regarded as heresy and both it and he are burned at the stake. Thinking that he may be alone in recognising his intentions for producing the work, he is uplifted by seeing a crowd at his execution, including ‘a little delegation from the village of Puerto’ (p. 116). Amongst these he notices one person whom he describes simply as ‘Nina (Goodness 2, Piety 5, Usefulness 10)’. The use of numerical scores to measure personality, professionalism and performance has rarely been expressed so meaningfully.

Reflecting on the messages in Wilder’s novel, it is a short step to consider the trend to measure the impact of professional learning and development (PLD), and the need to quantify (often for reasons of accountability) the benefits of particular models of PLD. This is pertinent to the ways in which we conceptualise, define and measure what we know as professional learning, and is relevant to the researchers, authors and readers of Professional Development in Education who strive to make sense of the complex elements of this aspect of professional education. So to what extent is it possible to measure professional learning? In its training form it is, by definition, measurable because ‘training’ should produce measurable outcomes. In terms of performance, we must be careful not to be reductionist, shaping learning inputs and outputs to measurable and behaviourist ‘professional standards’. In terms of impact on pupil learning, it is often relatively straightforward to simplify the transfer of professional learning activities into classroom outcomes; but ignoring the social, personal, professional and contextual complexities will inevitably lead to misleading interpretations, which is why there is no ‘magic bullet’ for PLD which transcends cultures and learning environments.

This is one reason why Professional Development in Education is such a valuable journal. It brings together the studies of education professionals and allows readers to discern the extent to which each study is relevant to their own situations. In her Editorial to volume 42 issue 2, Sue Swaffield (2016) comments that there are many ways of categorising and grouping articles on professional learning, including on the basis of geography, methodology, impact and scale. In methodological terms, she identifies ‘an array of research approaches and methods of generating and analysing data, including evaluation, case study, experiment, narrative, questionnaire, interview, critical discourse analysis, documentary analysis and statistical tests’ (p. 175) and applauds the ‘eclectic mix’ of articles which is likely to appeal to the different interests of readers, researchers, practitioners and policy-makers. The current issue of Professional Development in Education has a similar eclectic mix and provides a strong argument for having articles bound together so that they can open doors to a variety of approaches to research in PLD. The mix contains many qualitative studies but also includes quantitative approaches with statistical analysis of results. Both are appropriate and although we would not endorse statistical interpretations of ‘educational goodness’ as Brother Juniper may have done, we do recognise that professional learning takes many forms, many of which are observable and quantifiable.

The first article presents an interesting insight into the movement to shift from ‘traditional’ to ‘innovative’ approaches in the teaching of mathematics in the USA, and in doing so is presented using statistical analysis. Riley Lloyd, Veal and Howell argue that:

Traditionally, mathematics is viewed as ‘an orderly, enduring set of facts and logic,’ and to problem solve, one must first master these facts … The traditional perception is that such mastery: leads to one correct solution that is obtained through a particular method, algorithm, or memorized procedure; and confirms mathematical knowledge. (p. 360)

When this view is combined with direct, teacher-centred pedagogy, mathematics becomes a static subject promoting:

mediocrity and inequity. [Students] reproduce a culture that fears mathematics and finds it acceptable to ‘not be good at mathematics,’ instead of ‘producing’ critical thinkers and problem-solvers. (p. 361)

The article continues by outlining the ways in which data on teachers’ beliefs and practices are collected and analysed, an approach infrequently used in this journal. However, the use of statistical analysis in the authors’ methodology provides a rational basis for their conclusions and the claim that:

This study … provides clear data that teachers: do not all begin PD programs with the same conceptions about mathematics, teaching mathematics and how they teach mathematics; and are not necessarily consistent in their normative beliefs and discursive claims. (p. 380)

The second article by Izadinia provides a very different approach to research by looking at the ways in which mentors influence the formation of ‘identity’ in new teachers through their interaction in the pre-service stage. Focusing on the relationship between the mentor and the student-teacher, the article has relevance in the field of in-service (rather than pre-service) professional development because of the messages regarding relationship and identity that serving mentors should heed for the mentoring process to be effective. In contrast to the first article, the author here uses interview and thematic analysis to draw conclusions on the ways in which mentoring affects teacher identity. They use metaphor as a tool to guide the sampled individuals to give an insight into the relationship between the mentor and mentee and, from this, draw out a number of key aspects that mentors should bear in mind. The use of qualitative methods in this study contrasts strongly with the quantitative style of the previous article, and putting the two together provides a fascinating insight into the ways in which different researchers approach similar aspects of PLD.

The geographical focus in the first two articles moves from the USA to Australia. The next two articles relate to work undertaken in Malaysia, the United Kingdom and Indonesia. The article by Jarvis, Bowtell, Bhania and Dickerson provides an insight into the ways in which collaboration and a co-constructive approach to curriculum development provides a stronger and potentially more effective programme of teacher education. Rather than focusing on the new programme itself, the article turns the spotlight on the collaborators – the education professionals who were able to develop a self-critical approach in co-constructing the new programme in cross-national settings. The discussion of education in Southeast Asian and western educational settings provides a very interesting and important context for understanding the core principles of the new programme. A particularly important perspective is that:

Whilst Kapoor (2002), in a comparison of dependency and postcolonial theory, considers the idea that in any postcolonial nation it is not possible to have a discourse that is unaffected by colonialism, a comparative rather than postcolonial focus is used in this article. (p. 404)

In any consideration of professional learning arising from a cross-cultural collaborative process, it is important to make this point explicitly rather than risk the perception of cultural dominance that has pervaded a number of curriculum development models in the past. The fact that the process impacted on professional learning and practice for both sets of collaborators is an important outcome, and the benefits of a collaborative approach are as significant for what did not take place (i.e. a transmission model) as for the successes of effective shared ownership of the process.

The next article by Zein also focuses on a Southeast Asian context (Indonesia) but gives a highly critical perspective of reasons for the perceived inadequacy of professional development programmes for the teaching of English in that country. Zein argues that the high demand from parents for English to be taught in schools has led to a rapid rise in the number of teachers of English and that ‘The employment of teachers with poor English proficiency and no relevant qualification is ubiquitous’ (p. 424).

An interesting overview is given of the provision of professional development in Indonesia, focusing especially on the ways in which decentralisation of education decision-making has impacted (in the view of the author) on the quality of teacher education and on the tensions between professional and local political policies. The study, drawing admittedly on a small sample but supported by a wide range of literature, is reported in language which is at times emotive for an academic article:

The findings do not only demonstrate the educational tensions presently occurring in the Indonesian context; they also add another grim picture to the problematic situations surrounding the implementation of in-service PD for elementary English teachers in Asian countries such as Japan, China, South Korea, Bangladesh, Vietnam and Taiwan. (p. 431)

It will be interesting to see whether a response is forthcoming from other educators in Indonesia to confirm or reject the conclusions reached in this article.

Action research, or practitioner inquiry, is a phenomenon which has gained credence and popularity in recent decades. The next two articles focus on this aspect of professional learning, the first in Norway and the second in the USA. The process of what is often generically called action research is complex and, for new teachers, potentially fraught with micro-political problems if the findings intrude on the policies of the school or education system in which the professional is based. The article by Ulvik and Riese does not seek to oversimplify the introduction of this model of professional learning and recognises that it may be a ‘complex and challenging process that needs guidance and facilitation’ (p. 442).

The study involves the use of first-order and second-order action research (or action research upon action research) and one of the conclusions reached was ‘how challenging it is to teach and do research simultaneously’ (p. 442), a point frequently overlooked in the advocacy of this approach. It is interesting to read how the Norwegian students in this study are incorporating the research element into their master’s programmes and the ways in which, ironically, this is seen as difficult to prioritise. The study acknowledges the important roles of the teacher educators and of the schools in enabling the research component to work effectively, and in its conclusion recognises that to become part of a sustainable process of professional learning, action research needs to be seen as more than a project in the pre-service stage.

The second article on the theme of inquiry, by Clayton and Kilbane, echoes many of the points from the Norwegian study. They too advocate the importance of inquiry as a tool for professional learning and acknowledge that the professional development support needed to ensure effective inquiry would be ‘multi-dimensional, ongoing and complex’ (p. 458). The findings include an overview of the practical and conceptual tools which emanated from participation in the inquiry process and the realisation that ‘the foundational concept of collaborative inquiry, that of self and peer learning, eluded many of our participants’ (p. 477) and that the ‘teacher as expert role is also at the heart of the struggle teachers are having with designing student inquiries – what does the teacher do when not playing the role of expert?’ (p. 478)

The final two articles in this issue focus on one of the fastest growing aspects of education today, the understanding of new technologies and the introduction of new technologies into the process of teaching and learning. The provision of professional development support to enable effective teaching of and through the new technologies is central to the credible position of technology as a formal part of the school curriculum. One element of this is mobile learning (mlearning) and the article by Crompton, Olszewski and Bielfeldt provides an insight into how effective professional development support may be structured to meet the emerging needs of students and teachers in the USA. Being able to use new technologies is not a recipe for being able to teach with and about them effectively, and the disconnect between technology and pedagogy, often attributed to a lack of effective professional development support, forms the focus of this article. mLearning is defined as ‘Learning across multiple contexts, through social and content interactions, using personal electronic devices’ (p. 483) and the importance of providing learning across multiple environments and subject area contexts is emphasised, forming an even greater challenge for those providing professional development programmes or seeking to engage in their own professional learning agendas. The data from the research undertaken are analysed and presented both statistically and qualitatively. A conclusion reached is that for such a fast-developing area ‘it would appear that there is no set model of what should be provided in mlearning training, but instead that ongoing support and coaching is the single greatest need of teachers implementing mlearning practices’ (p. 497) and that it will be necessary for all engaged in mlearning ‘to walk the line between the fundamental and futuristic’ (p. 499).

The final article, again from the USA, is a Viewpoint piece by Boyd and Sampson. This continues the theme of integrating digital technologies with educational practice, this time in higher education. They comment that there is a ‘dissonance in identities between creative practitioner and educationalist’ (p. 503). They identify the need to allow ‘structured migration between technologies according to context and task’ both in the pedagogical aspect of teaching and in the approaches to professional learning. Considered within the context of art and design education, the article discusses the need to promote digital confidence in creative settings. They conclude that the confidence to experiment with digital technologies is important, but this must take place in ‘a supportive context where collegiate dialogue on educational appropriacy is possible’. The reader must ponder on the extent to which any measurement of either of these two factors would be appropriate or relevant.

Ken Jones
Managing Editor