Ethically scaling up interventions in educational development: a case for collaborative multi-sited ethnographic research

ABSTRACT

Educational interventions are often administered at scale in diverse settings as part of international development programmes. Their implementation is subject to a linear process that begins with finding out ‘what works’ at a local level, frequently through the use of randomised controlled trials, and continues with rolling out the intervention to the whole population at a national or even transnational level. This process often fails to consider the role cultural, political, and historical factors play in the perceived success of the local intervention, which can compromise both the impact and the ethics of at-scale implementation. To help address this issue, this paper argues for a definition of scalability that incorporates the ethics of the practice of scaling. It points to the potential of collaborative multi-sited ethnographic research to identify nuanced understandings of the different ethics systems endogenous to individual sites of implementation, in lieu of the universalising notions of ethics that are embedded in mainstream, linear notions of scalability. In so doing, it makes the case for multi-sited critical ethnography as a methodology of choice in researching the scalability of interventions in the context of development projects in the ‘Global South’.

KEYWORDS:

• Educational development
• scalability
• ethics
• global south
• policy transfer
• educational borrowing and lending
• context
• multi-sited ethnography

1. Introduction

In my work as a development practitioner, I have often heard the term ‘scale’ described as the holy grail of development. If we can find a solution for issues of low enrolment, low educational quality, or whatever problem we may be working on and are able to apply it not just locally but nationally or even globally – that is, at scale – we will have succeeded (cf. Milat et al. 2013; Richard 1996; Samoff, Dembélé, and Sebatane 2011). This narrative, while highlighting the urgency of educational development challenges and the need to look for solutions at the local level, leads to an overly simplistic understanding of the process of scaling in which ‘scaling up’ becomes synonymous with ‘replication’. In other words, a local intervention deemed successful (frequently using quantitative techniques such as randomised controlled trials, or RCTs) is seen as a magic bullet capable of solving the same problem across a wide range of contexts.

Scholars in the tradition of postcolonial studies, subaltern studies, anthropology of development, critical sociology, and other areas have long pointed to this issue. They often depict the idea of solving development challenges at scale as neocolonial, insensitive to local cultural realities and conflicts, and therefore fundamentally problematic or even doomed to failure (Escobar 1997; Lewis 2012). Many interventions in the history of educational development have no doubt failed or been implemented in culturally insensitive ways as a result of the universalistic, essentialising conception of scaling that many development practitioners, knowingly or unknowingly, espouse in their work.

Yet, in light of the massive challenges educational development around the world is facing today, to dismiss the concept of scale altogether would be irresponsible. It is difficult to imagine these challenges being solved by a large number of localised interventions that target the local needs of every culturally distinct population in the world. Such an approach, in the tradition of Schumacher’s (1974) ‘small is beautiful’ dictum, certainly seems to render impractical the involvement of affluent donors. This would be an ethically problematic outcome, considering the sheer need for educational development resources and the history of colonialism and exploitation on which the wealth of many of these donors is founded.

Instead of dismissing the idea of development at scale, I propose a redefinition of ‘scale’ and ‘scalability’ and a corresponding shift in the methodology of choice in researching these concepts.¹ While these two terms are not fully synonymous (‘scale’ here pertains to the desired magnitude of impact, whereas ‘scalability’ refers to the potential to reach this magnitude), they are underpinned by the same set of ideas (growth, translation, transfer) and fraught with the same challenges (including imperialist histories). I argue that, if we are to address the historical baggage attached to these concepts, we need to work toward a ‘thick description’ (Geertz 1973) of them, one rooted in culturally informed understanding of the translation process and the accordant tensions between form, content, and the ethicality of outcomes.

The ethics of scaling can only be understood ethnographically and through the co-production of knowledge in the field. This means that, instead of imposing an exogenous, purportedly universal system of ethical values, we must grasp some understanding of the multitude of local meanings – often conflicted meanings – of ethics across the diverse communities spanned by a scaled intervention. Knowledge about the ethics of scaling therefore needs to be co-produced by the researcher and the intended beneficiaries of scaling efforts, in the tradition of collaborative and reciprocal ethnography (Lassiter 2001, 2005; Tedlock 1991).²

This article is divided into four main sections. In the first section, I propose the idea of a thick description of scale that incorporates the ethics of scaling. In the second section, I review several bodies of relevant literature and show that the proposed definition is consistent with previous scholarly critiques of dominant concepts (‘thin descriptions’) of scale and scalability. I also show that, while much recent scholarship about comparative education has acknowledged the importance of context in researching education interventions that span diverse settings, this scholarship has not recognised the unique contribution a context-driven methodology – that is, ethnography – can make to our understanding of different systems of ethics. The penultimate section discusses some of the methodological implications of using multi-sited ethnography to study scaling and outlines several conceptual tools and potential research questions that seek to generate knowledge that would depolarise some of the debate around scaling. The conclusion revisits the idea of considering the agency of the development subject when shaping research into scaling.

2. What is scalability?

The term ‘scalability’ refers to the idea of expanding programmes to reach more beneficiaries. Some practitioners and scholars use the concept interchangeably with ‘transferability’ and educational ‘borrowing’ and ‘lending’ (Phillips and Ochs 2004; Rappleye 2012; Steiner-Khamsi 2004, 2012; Steiner-Khamsi and Stolpe 2006). Much of the literature on these concepts focuses on a linear transfer of knowledge from one place to another. One definition of scalability is rooted in these ideas and is resonant with the World Bank’s policy accounts of scaling (Hartmann and Linn 2008): ‘The ability of a[n] […] intervention shown to be efficacious on a small scale and or under controlled conditions to be expanded under real world conditions to reach a greater proportion of the eligible population, while retaining effectiveness’ (Milat et al. 2013, 289). The focus on expanding the number of beneficiaries and/or geographies serviced by a programme (Uvin and Miller 1994) implies a unidirectional transfer of interventions from site to site – a process Soini, Pietarinen, and Pyhältö (2013) describe as ‘mechanistic’.

Fisher (1993) brings more nuance to the definition by recognising the distinction between scaling up and scaling out. According to Fisher, scaling out refers to expanding the constituencies of an intervention (project replication), whereas scaling up pertains to a programme’s ability to effect political transformation by tackling the ‘root consequences of underdevelopment, and not its manifestations’ (Uvin and Miller 1994, 2), a view shared by Korten (1987) in his advocacy of ‘third-generation’ NGOs and by Clark (1991) in his book Democratizing Development. While this distinction makes visible some of the nuances of politics and culture involved in the concept of scalability, it does not fully illuminate the ethical values that underpin the scaling process.

To arrive at an even more comprehensive conceptualisation of scale, these two aspects need to be complemented by a third: ethics. Much of the world’s scaling efforts occur in the context of international development efforts concentrated in the ‘Global South’ and are carried out by international organisations, NGOs, governments, and, increasingly, private partners. The term ‘Global South’ in this context is not restricted to the least developed countries; it refers instead to ‘those communities/populations whose circumstances (economic, cultural, political, technological), when compared to the rest of the population in that territory, are highly precarious and marginal’ (Robertson and Komljenovic 2016, 595). In many of these regions, the concept of development comes with imperial baggage. The colonial histories of these countries reveal that Western empires often used ideas of modernisation and progress to expand their reach.³ The concept of the civilising mission, associated in particular with the British Empire, was used to justify the exploitation of colonies for the benefit of the imperial metropole. Colonialism was advanced under the guise of a moral enterprise on the premise that colonies were ‘backward’ and needed ‘civilising’ in order to ‘catch up’ with the ‘developed’ world. At the heart of colonial education was the premise that Western knowledge was universal and superior to all other knowledge systems, and this epistemic hierarchy was seen to be inherently scalable across the Empire.

This historical context makes scalability in the ‘Global South’ not only a contested project amongst the various groups most affected by scaling but one in which we arguably need to be as much concerned with the ethics of the practice of scaling as with its effectiveness. Bringing in outside interventions is not always a question of choosing the most effective or ethical solution but a political and economic imperative. As Steiner-Khamsi (2012) argues,

[p]olicy borrowing in poor countries is to the education sector what structural adjustment, poverty alleviation, and good governance are to the public sector at large: a condition for receiving aid. As a requirement for receiving grants or loans at the programmatic level, policy borrowing in developing countries is coercive, and unidirectional. (5)

This argument often resonates with the substantive moral and political concerns expressed within postcolonial and critical development studies about the dynamics of the international and global economic forces at play in shaping development aims and outcomes (Baaz 2005; McEwan 2009; Morris and Spivak 2010), including the limited agency those living in the ‘Global South’ have in shaping their own policy landscapes (Mitchell 1988; Wainwright 2008). The question of what is at stake and for whose benefit is often blurred in such global public policy debates. Moreover, these power dynamics are often underpinned by assumptions of the universal relevance of interventions – usually solutions designed by ‘experts’ in the ‘Global North’ – which may not hold true if contextual factors that affect scaling are taken into account.

Considering the historically limited agency the development subjects have in defining the goals and parameters of projects designed to help them, as well as the vast economic inequalities and power differential between the ‘Global North’ and ‘Global South’, ethics must be a key concern for any educational development project. In the schematic depiction below, which is not meant to represent a binary but rather a continuum, I use the deliberately vague terms ‘ethical’ and ‘unethical’ to demarcate the possible directionality of scaling practice. This is not to say that the ethicality of a particular scaling process cannot be investigated with rigour. The point is that, unlike scaling out, which can easily be assessed quantitatively, the ethical dimension of scaling cannot simply be measured. The criteria used to gauge the ethicality of scaling will differ from site to site, and generating these criteria will require the co-production of knowledge between the researcher and the researched in the context of collaborative, multi-sited ethnographic studies. The notions of ‘ethical’ and ‘unethical’ will therefore have different meanings in different contexts. Crucially, including a normative definition of ethics in a theory of scalability would be anathema to the idea of ethical scaling.

When we bring these threads together, a three-dimensional model of scalability emerges that consists of the dimensions of replication (scaling out), transformation (scaling up), and ethics. We might imagine this model as a coordinate system with three axes:

In Figure 1, the vertical axis represents scaling out, which is the expansion of interventions along the continuum from the local micro level to the transnational. The horizontal axis stands for scaling up, or the extent to which an intervention carries a political charge capable of effecting a social transformation. The diagonal corresponds to the ethical dimension. Many development programmes would, arguably, aspire to scale in terms of achieving greater reach and transformation while maintaining or improving the ethics of their engagement. Such scenarios are in the triangular area highlighted in the diagram.⁴ An ethnographic methodology can help us understand the extent to which the scaling process of an intervention might be headed in the desired direction along the different axes – not through an externally devised evaluation framework, as is often the case with development projects, but through an immersive, iterative, context-driven process at the grassroots level of uncovering the very criteria used for evaluation.

Figure 1. The three dimensions of scalability.

While this is a normative framework, its normative dimension is the outcome of a collaborative effort in which the study participants – in particular those with historically limited ability to have their views heard by the powers that be – shape the definition and the indicators of the ethicality of a practice. As the next section illustrates, the customary approach in both conceptual and empirical studies of scaling is far from this, despite scholars’ frequent calls to add more depth to investigations of scale and scalability.

3. What are the advantages of a ‘thick’ over a ‘thin’ description of scalability?

The issue of scale and the related concepts of transfer, translation, and policy borrowing have received a lot of attention in the comparative and international education literature. Much of this literature is critical of the lack of nuance and depth in studies dealing with education programmes implemented at scale (what we can refer to as thin descriptions of scalability). However, it does not explicitly call for ethnographic approaches to these questions. In this section, I show that the concept of a thick description of scalability addresses many of the concerns raised in this literature.

3.1. The limitations of a thin description of scalability

One of the most consistent voices for deeper engagement with the concept of scale is that of Coburn (2003), who in her article Rethinking Scale calls for studying scale in greater depth. The methodological implications of such a shift in focus call

into question the degree to which classroom implementation can be assessed using survey methods alone. Capturing depth may require in-depth interviewing and classroom observation, refocused on such indicators as the nature of instructional tasks, discourse patterns in the classroom, and teachers’ conception of knowledge and learning. (5)

Such indicators are clearly affected not only by the intervention being scaled but also by contextual factors at the site of implementation. This view resonates with the work of scholars who have studied the execution of large-scale reforms, such as Soini, Pietarinen, and Pyhältö (2013), who write:

In large-scale school reforms the goals of development work are seldom fully achieved … [I]n practice the reforms are rarely implemented in terms of multidimensional learning processes, but rather, they are seen merely as self-evident mechanistic tools to attain educational goals. (71)

This literature illustrates that to gain insight into expanding the reach of ‘multidimensional learning processes’, it is necessary to consider scalability as a multidimensional phenomenon, rather than to focus merely on the quantifiable outputs of scaling interventions. As Robertson and Dale (2015) point out, ‘[a] typical form of output is a qualification; something “tangible” to “show” results from the experience of schooling … By contrast, the idea of outcomes refers to what outputs are produced for, and what they might enable’ (162, emphasis in original). By focusing on outputs, ‘we [often] have description rather than analysis, [which] lacks any theory of agency’ (158). In other words, our understanding of the dynamics at play in scaling processes, their ethical aspects in particular, is often limited because of the conceptual frameworks and definitions with which we operate.

As a result, empirical studies that examine educational interventions implemented at scale across diverse national, cultural, and political landscapes are often unable to make a significant contribution to our understanding of the ethics of scaling. As Coburn (2003) points out, ‘most educational research that focuses on scale has tended to define it in unidimensional ways, involving solely or predominantly the expansion of number of schools reached by a given reform effort’ (3). Harwell (2012) concludes similarly that ‘empirically supported theories/models or strategies and practices linked to successful multisite/scale-up studies are not available’ (34) in education literature. Mukhopadhyay and Sriprakash (2011) point out that ‘[t]here is still a paucity of detailed research in the field of international and comparative education examining how policies and programmes are constituted in local settings, especially in contexts of significant political, social and economic change’ (314). Klingner, Boardman, and McMaster (2013) recognise that this gap in research is caused in part by the disconnect between researchers and practitioners: ‘these different stakeholders tend not to share the same goals, and the result is an incoherent and disconnected education system that preserves through its division of labour a pronounced gap between the worlds of research and practice’ (197).

Given the dominance of economic thought in much of the scholarship about international development, it is not surprising that studying political and cultural factors, which is crucial to examining the ethics of scaling, apparently has not always been given due consideration in the literature outside of anthropology.⁵ According to Stephens (2007), ‘it is the denial of culture – and I would argue the power of that culture and the corresponding hegemony of so-called “culture-free” economics – that has led to much of the failure of development projects during the past 30 years’ (41). Consequently, ‘[p]roblems of implementation [are] viewed as “barriers” rather than deeper, cultural forms of resistance to what is going on’ (41). In their framework for cross-cultural education administration, Dimmock and Walker (1998) argue that ‘it is the transfer and mobility of theory, policy and practice [that] needs to be more “culture sensitive”’ (37) or culturally relevant.

The consensus in the literature seems to be that studying the varied, complex, and sometimes contradictory cultural and political processes in particular contexts in order to better understand the context of scaling up interventions can improve our understanding not only of what works in educational development but also of what is or may be ethical.

3.2. Toward a thick description of scalability

The literature critical of the ‘thin’ descriptions of scalability suggests that we need to develop a more nuanced ‘thick’ description suitable for educational environments and cross-cultural implementation practices. A study of scalability needs to focus on both the processes and outcomes of its application, rather than merely on the outputs. Filling these gaps requires a conceptualisation of scalability as a ‘project of a particular kind (precisely, a scale-making project)’ (Ferguson 2011, 201), and may provide a more comprehensive and culturally relevant picture of scalability as it relates to time and place.

The conceptualisation of scalability as a three-dimensional phenomenon that encompasses important ethical considerations and has the potential for political transformation is in line with the increasing attention comparative education scholars and educational development practitioners are paying to the context of interventions (Cowen 2006; Vavrus and Bartlett 2009). Over the last two decades, we have seen article-length studies that explored the implementation of up-scaled interventions (Croft 2002; Dyer 1996; Kanu 2005; Prophet 1995; Tabulawa 1997), as well as theoretical models of policy transfer and scalability that have recognised the role of contextual factors in determining the outcomes of scaling processes (Phillips and Ochs 2004; Rappleye 2012; Steiner-Khamsi 2012). These studies have broken away from the logic behind linear theories of change that rest on the assumption that outside intervention alone can lead to internal transformation; in other words, they have treated sites of implementation not as blank slates to be filled with the knowledge of development ‘experts’ but as loci with existing cultural, political, and social realities that greatly affect the ability of any given project to ‘scale successfully’. This has been a necessary but insufficient step in developing a thick description of scalability, which is key to improving our understanding of endogenous systems of ethics and also has the potential to defuse some of the historical baggage of scalability that is rooted in colonial hegemony.

Several scholars have gone beyond recognising the role context plays in shaping interventions. Sobe and Kowalczyk (2013) contend that ‘too often in the field of comparative education the issue of context is treated as a “matter of fact” when instead context should be envisioned as a “matter of concern”’ (6). They suggest that, rather than examining context prior to and separately from a study’s primary research questions and using it merely to ‘stabilise’ the research project, researchers need to focus on developing categories and analytic topics that are related to context, as these themselves are inevitably embroiled in power relations.⁶ Insofar as we can think of ethics as an analytic category, Sobe and Kowalczyk echo my argument about the need for the primacy of context in devising translations of the ethics systems endogenous to implementation sites. It is thus possible to interpret their argument as an implicit nod to collaborative ethnographic approaches that seek to empower research participants, rather than to impose external analytic categories for value systems and ethics.

The idea of the primacy of context is central to that of the comparative case study developed by Bartlett and Vavrus (2017). This methodology, with its simultaneous attention to global, national, and local dimensions of policy flows and its commitment to the principles of multi-sited ethnography, shares a good deal of conceptual terrain with the approach advocated in this paper. Yet, as Bartlett and Vavrus (2014) acknowledge in their discussion of how their framework was used in empirical studies of learner-centred pedagogies in Tanzania, ‘[w]hile affiliated researchers considered themselves as the “instruments of research” and their experiential, embodied knowledge as central to their learning, they nevertheless did not experience the level and extent of immersion that is the hallmark of ethnography’ (138), in part due to the limited amount of time they spent on the project. This is one reason why ‘this specific project and other studies that have used the approach may be more properly described as qualitative case studies’ (138). Possibly indicative of a climate not conductive to ethnography (as I discuss in the Conclusion), this admission is not merely a sign of the gap between the idealism of conceptual scholarship and the practical constraints on empirical fieldwork many researchers face; it also points to differences between the notion of a comparative case study and the ethnographic approach to the study of scalability I argue for in this paper. Researching value systems endogenous to implementation sites – the primary concern of the approach outlined here – allows for much more focused study designs than the vertical case study. The ethnographic approach rests on the assumption that the ethics of scalability lie at the core of the tension between the critics and proponents of scale. If this assumption holds true, creative collaboration can occur between these two camps when both recognise that, while interventions (or at least their components) may have elements of universalism, this is hardly ever true of value systems. Zeroing in on ethics and prioritising depth in one area of inquiry over breadth across many, as with vertical case studies, can make a true ethnographic approach not only more feasible but more effective in identifying the enablers and constraints of scaling.

A thick description of scalability must include an understanding of the cultural norms and practices, and the associated patterns of social and political life, in the sites of both the project design and the implementation. As Hobart (1993) points out in his introduction to An Anthropological Critique of Development, the dominant development paradigm often sees local culture and indigenous knowledge as obstacles to progress: ‘Claims to knowledge and the attribution of ignorance are central themes to development and remain seriously under-studied’ (4). The culture of the communities targeted by educational development interventions is not all that matters to the study of scalability; according to Lewis and Mosse (2006), understanding the culture of the brokers and ‘translators’ of development policies – those acting in the ‘unscripted inter-institutional, intercultural brokerage roles’ (16) – is also crucial to examining the effectiveness and limitations of development interventions. Developing a thick description of scale in development therefore requires us to also incorporate elements of institutional ethnography into research design.

As the differences presented in Table 1 reveal, a thick description of scalability is a complex, multilayered idea that sets an ambitious agenda for the study of scaling processes. An ethnographic approach to studying education interventions across multiple sites is uniquely positioned to address this agenda. Unlike impact evaluation studies that assess the performance of individual agents and contexts operating within various development projects, the ethnographic method is suited to consultation between the various actors involved in designing and implementing development interventions. As Lewis and Mosse (2006) argue, since ethnography

is not constrained to privilege authorized (instrumental) interpretations, it can throw light on areas of development practice that are hidden or silenced by policy, but that are critical to understanding how events actually unfold in particular settings and why interventions do or do not work. (15)

Table 1. Comparison of ‘thin’ and ‘thick’ descriptions of scalability.

Thin description	Thick description
Scaling as linear transfer of knowledge	Scaling as a messy, nonlinear process
Context as a ‘stabilising variable’	Context as the primary driver of research
Universalising notions of ethics	Site-specific systems of ethics
‘Indigenous knowledge’ seen as irrelevant at best, an obstacle at worst	Indigenous knowledge recognised as an important contextual factor
Focus on education ‘outputs’	Focus on education ‘outcomes’

To understand the need for a multi-sited ethnographic approach, it is useful to consider the field as a social construct. We can conceptualise it as a community with socially and culturally divergent landscapes and regionally specific challenges. As Nadai and Maeder (2005) argue, ‘the field of sociological ethnography cannot be found somewhere out there, but is constructed by the researcher’ (4) as the project focuses on studying a sociological concept (scalability), and thus the ‘locus of study is not the object of study’ (4).⁷ Studying scalability through the prism of ethnography can be seen as being on the continuum between sociological and cultural ethnography; such an approach would consider the interventions’ interactions with existing cultural and political landscapes across multiple sites.

Incorporating multiple sites is important due to the nature of scalability. When the subject is ‘mobile and multiply situated’ (Marcus 1995, 102), as is scalability, a multi-sited, cross-national approach is necessary for the analysis. Rather than being constrained to the accommodation–resistance dichotomy, multi-sited ethnography is able to examine subtle continuities and ruptures within the cultural and political flows of scalability. It also offers a more complex account of its processes and effects, and therefore becomes the mechanism for translation as well as potentially functioning as an assessment tool of successes and failures of scaling in development. As Marcus (1995) writes, ‘The key question is perhaps: What among locally probed subjects is iconic with or parallel to the identifiably similar or same phenomenon within the idioms and terms of another related or “worlds apart” site?’ (111). In answering this question, Marcus argues that ‘the work of comparative translation and tracing among sites’ is ‘basic to the methodology of multi-sited ethnography’ (111). The idea of ‘translation’ – defined by Cowen (2009) as ‘the shape-shifting of educational institutions or the re-interpretation of educational ideas’ or the ‘chameleon process’ (323) – indeed bridges the conceptual framework of multi-sited ethnography with comparative education. Seen through the prism of these ideas, scaling becomes a nonlinear, context-driven process that can be assessed in diametrically different ways, depending on stakeholders’ frames of reference. In other words, scalability cannot be assessed prima facie, it must be understood as a project of cultural, economic, and political translation.⁸

4. What conceptual tools can be used to build a ‘thick description’ of scalability?

Having presented the benefits of a thick description of scalability, I will now outline principles that can be used to operationalise this idea in research design. Every intervention and set of contextual factors call for an individualised approach, thus there is no single way to study scalability. My goal, therefore, is to identify overarching concepts that can be helpful in designing studies that aim to get beyond thin description. The conceptual tools identified in this section are neither exhaustive nor prescriptive; the reader can think of them as suggested additions to the register of methodological approaches to studying interventions implemented at scale. The ideas outlined here are intended in particular to help facilitate a process of co-producing knowledge about the ethics of interventions implemented at scale, with a particular focus on understanding endogenous value systems unique to individual implementation sites. The section revolves around two analytical toolsets: micro-level concepts that illuminate the on-the-ground dynamics of a particular intervention, and macro-level research questions that build on these concepts by formulating possible strands of research on the ethics of scalability that are in line with the definition suggested in Section 2.

4.1. The micro level: the scaling object and support factors

To examine the ethics of scaling, we first need to develop a conception of the interface between the intervention being scaled and the contextual factors that shape that implementation. To do this, we need to distinguish between two components of an intervention. On the one hand is an element of intervention intended to remain unchanged as scaling occurs; we may call this ‘the scaling object’ (Mickelsson, Kronlid, and Lotz-Sisitka 2018).⁹ An example might be a pedagogical approach to teaching environmental education in schools that stresses individual action over abstract concepts and is intended to appeal to children all over the world. However, this element's success depends on other components it can adapt to each scaling site; we may call these ‘support factors’. Environmental education teachers’ willingness to consider changing their approach is one example of a support factor.

Effective scaling requires identifying a scaling object whose usefulness transcends a single site and then replicating it at another site with a high degree of fidelity to (i.e. consistency with) the original object. It is worth noting that the fidelity of the scaling object does not imply that it is static. On the contrary, many successful scaling objects scale well precisely because they contain mechanisms for adapting to local conditions. For instance, the environmental education intervention might include a teacher-training programme that is applied consistently in all sites (high fidelity) but comes with an internal curriculum that changes from site to site according to local needs (high adaptability).

The support factors an intervention requires to be effective in sites of scaling need not be identical to those at the original site, particularly if the scaling object is adaptive to different circumstances. If teachers in a certain scaling site are not willing to change their approach to teaching environmental education, for example, the teacher-training programme might harness these teachers’ motivation to participate in professional development and use the training sessions to increase their motivation to explore alternative pedagogical approaches.

The object of scaling out might differ from that of scaling up. Whereas the former is typically a key element of an intervention, the latter, being at the core of social transformation, is usually a political goal or value. For example, in an international education reform that relies on pedagogical innovation to accomplish the goal of contributing to ethnic reconciliation, the scaling object of scaling out will be the specific practices introduced in different communities, which depend for success on the support factors present in these communities, whereas the scaling object of scaling up might be appreciation for ethnic diversity, toward which the reform aims to direct the learners.

The focus on randomised controlled trials among development scholars and practitioners can be seen as a fixation on the idea of scaling objects (cf. Banerjee and Duflo 2012; Banerjee et al. 2007). By rigorously measuring the impact of isolated interventions, RCTs hope to identify solutions that work across different contexts. However, as Cartwright and Hardie (2012) have pointed out, RCTs typically have limited external validity; an intervention that is effective in one site is not guaranteed to work elsewhere. It is therefore imperative to identify the support factors necessary for an intervention to produce the desired effect. When assessing whether an intervention can lead to a similar outcome elsewhere, it must be determined whether the same support factors exist and play the same causal role in both locations.

In this framework, support factors are seen as discrete; their presence in one location is not thought to affect their presence (or the causal relationships between them, interventions, and outcomes) in others. However, Appadurai (1996) points out in his seminal work, Modernity at Large, that globalisation has led to the emergence of cultural flows which are ‘not only reconfiguring local cultures in the periphery but also challenging the cultural certainties and fixities of the Metropole’ (42). This work has been central to understanding cultural landscapes as highly complex entities which shift in relation to wider global changes. Scalability of education interventions is an example of a cultural flow that is multidimensional across all five ‘scapes’ Appadurai identified: ethnoscapes, technoscapes, financescapes, mediascapes, and ideoscapes. The support factors that make the upscaling of interventions successful are not discrete phenomena; rather, they are located in complex cross-cultural discursive flows. Support factors are therefore – perhaps in contrast to many scaling objects – unlikely to be measurable in quantitative terms.

The process of scaling can be visualised in terms of scaling objects and support factors. Figure 2 shows several of the many possible scenarios of scaling. In the uppermost scenario, the scaling object retains its fidelity to the original intervention, as represented by the size of the black circle in the illustration, but the site of scaling does not offer support factors relevant to the intervention. In this example, an inappropriate site was chosen where scaling up is unlikely to succeed even if scaling out is successful, because the transformative potential of a scaling object can only be realised in the presence of appropriate support factors. In the middle scenario, the fidelity of the scaling object decreases proportionally to the available support factors. In this case, the success of scaling will depend on whether or not the decrease in fidelity and the fewer support factors render the transformative potential of the intervention ineffective at this particular site. In the final scenario, the fidelity of the scaling object decreases, despite there being a greater abundance of relevant support factors at the target site. Such a scenario is likely to result from deficiencies on the part of the implementation agent, such as low institutional capacity, inadequate monitoring, corruption, and the like. While such factors – as well as the importance of choosing a suitable site, as demonstrated by the first scenario – are undoubtedly important, the interface between the scaling object and its support factors is key to the intervention’s success in all these cases. In order to isolate the phenomena happening within this interface as the subject of study, the analytical framework that follows assumes that scaling objects are implemented with high fidelity across sites and that the appropriate support factors are present at each site.

Figure 2. Different scenarios of transformations undergone by scaling objects.

While the simplicity of the framework outlined so far might seem appealing, this comes at the cost of using an externally imposed definition of success and therefore also using exogenous criteria to evaluate the ethicality of the scaling process. Social transformations, however, do not always serve ethical goals. Often, they might appear to do so from the perspective of the outside implementation agent while in fact being considered unethical by their supposed beneficiaries. An example of such an intervention can be seen in scenario (D) in Figure 3. (In this diagram, the x-axis represents time and the y-axis the respective phenomenon being examined – scaling out, scaling up, ethics – as depicted in the three-dimensional model in Figure 1.) This intervention, despite its success in scaling out and a modest ability to scale up, saw its ethics decline. In the ethics diagram at the bottom, the dotted line represents a ‘threshold of acceptable ethics’¹⁰ of implementation, which this intervention crosses. We can label this scenario ‘unethical scaling’. In scenario (C), ethics remains unchanged, as does the number of beneficiaries. The only change in this case is the scaling up of the intervention; that is, even though the number of people affected does not change, the intervention’s ethics remain constant and it successfully effects social transformations that help to remove underlying causes of deprivation at the target site. We can label this scenario ‘invisible scaling’. Scenario (B), on the other hand, saw an increase of beneficiaries and an improvement in ethics, but its transformative potential was not realised. Many development programmes considered successful would follow this trajectory: by focusing on the number of people affected and the ethics of the implementation, the larger concern about eliminating the need for development interventions in the long run is obscured. We can label this scenario ‘the quick fix’. In contrast, scenario (A) not only saw the number of beneficiaries expand and its ethics improve, it also led to a corresponding realisation of its transformative potential which contributed to resolving the underlying causes of deprivation among the communities served. Arguably, such interventions are rare, and while demanding that each intervention ‘do well’ along each of the three dimensions of scaling might seem an unachievable aim, it is worth striving for.

Figure 3. The different scaling trajectories of interventions.

Crucially, the idea of ethics in this model is not based on any externally devised criteria. Whether an intervention is moving in a more or less ethical direction as it scales and whether it passes the threshold of acceptable ethics – and, indeed, where this threshold is located – are subject to the value systems stakeholders share at individual implementation sites. Therefore, before assessing the ethicality of scaling, it is first necessary to define what ethics means at each individual site. This in fact is the main reason an ethnographic approach must be used, as the immersion, openness, and flexibility it offers is needed in order to understand what different groups of people deem to be ethical.

Given these theoretical assumptions, we can pose key conceptual questions related to scale and scalability: In what ways can the programme implementers ensure that scaling out the intervention is in the service of scaling it up in an ethical way, as ethics is understood by local stakeholders? In other words, how can increasing the geographical reach of an intervention help to resolve the underlying causes of underdevelopment in an ethical manner? Exploring these questions through the framework of a thick description of scalability is the subject of the next section.

4.2. The macro level: capturing the depth of scalability through ethnographic research

It is imperative to recognise that the answers to the ‘big questions’ about scalability are subject to a complex set of interactions between the interventions being scaled and the sociocultural, political, economic, and historical context of the scaling sites. To assess the transformational potential of an intervention at the local and societal level, we need to understand what Tsing (2005) calls social friction – the space in which large ideas meet small spaces. It is in this realm that we gain insight into the factors that enable and constrain the upscaling of programmes; too much ‘friction’ leads to resistance, too little to disengagement.

Given this complexity, it is helpful to break down the overarching question into sub-questions that focus on five key areas (see Table 2). Rather than including ethics as a separate category – which could be seen as tokenism – ethical concerns underpin questions across all five areas. This list of questions is not meant to be exhaustive, nor is it assumed that any given empirical study can answer every question. Rather, the goal is to use these clusters of questions as entry points into the multidimensionality of scalability, and then to let the unfolding research process determine which questions become the main foci of the study and in what ways they might need to be refined in order to connect with the empirical realities of individual research sites.

Table 2. Research questions directed at different aspects of scaling.

Aspect of Scaling	Research Questions
1. Scaling out: context, comparison, solutions	In what ways do the two (or more) targeted sites represent elements of convergence and divergence in the transfer, translation, and implementation of the project? How do stakeholders understand and account for these differences across sites? What are the potential cultural conflicts and interpretations associated with the border politics of transfer?
2. The transformative (upscaling) potential of the intervention	What ethical and geopolitical assumptions did the implementation organisation make about scalability in the process of scaling the intervention? What notions of ‘development’ underpin these assumptions? What are the economic imperatives?
3. The underlying causes of deprivation	How is the history of development understood and conceptualised by stakeholders living in the sites of scaling? How do these conceptualisations vary or converge across different groups of stakeholders? In what ways do they reflect the goals of equity and social justice, as understood by research participants? Does the intervention target any of the issues of deprivation identified by people living in the respective areas?
4. The outcomes of the intervention	Do local residents perceive the intervention as adding to, detracting from, or not affecting existing efforts to tackle underlying causes of deprivation in their communities? Do such perceptions vary or converge across different stakeholder groups? Do the supposed beneficiaries targeted by the intervention in these sites emerge with a changed understanding of the world? If so, can such learning outcomes be described as contributing to the more abstract but ultimately ethically significant forms of social transformation in the sites of scaling and beyond?
5. Global outcomes of scaling	Does the comparison of the impact of the scaled programme across scaling sites show any patterns of cross-national convergence/-divergence?

Some of these questions can be asked about the implementation of an intervention in a particular site, while others require examining the intervention in the context of more than one site. As shown in Figure 4, the underlying causes of deprivation and intervention outcomes are best analysed for each site separately. On the other hand, the nature of scaling out and the transformative potential and global outcomes of scaling are research subjects that require insights from more than one site, since these questions are concerned with the trajectory of scaling objects as they move between sites. Even though it may be possible to investigate the transformative potential of scaling a particular intervention separately for different sites, the notion of scalability implies a degree of universalism of the scaling object; therefore, comparable social transformations would be expected to occur in different sites of scaling. This model assumes that the transformative potential of the intervention does not change according to the geography of implementation; rather, the presence (or lack) of appropriate support factors at the site, as well as any variation in the underlying, site-specific causes of deprivation, would account for any differences in the extent to which the transformative potential is realised at any given site.

Figure 4. Mapping research questions aimed at developing a thick description of scalability.

This model does not incorporate several considerations relevant to the scaling process in its framework, including the implementing organisations’ institutional capacity to administer programmes at scale or to monitor and evaluate aspects of the intervention. By assuming that the scaling object can be replicated with high fidelity to that of the original intervention, the model shifts its focus away from institutional concerns about implementation agents, which are a frequent subject of discussion among development practitioners, and toward the interface between the scaling object and the cultural, political, economic, social, and historical context of the scaling sites. The argument made is that, by focusing on this interface, it is possible to gain valuable insights into both the scaling up and the ethics of scaling – two crucial dimensions of scalability that, while consistent with the increased importance attached to context in both academia and development practice, have received less attention than scaling out.

5. Conclusion

The very notion of scale invokes the metaphor of quantification. If a project or an idea ‘makes it big’, the questions that immediately present themselves – such as ‘how big?’ – often have the expectation of a numerical answer. It might therefore seem counter-intuitive that the model presented in this article calls for an ethnographic approach. The goal is not to dismiss quantitative indicators of scale, as they can be particularly helpful when it comes to understanding the dynamics at play in scaling out; rather, the goal is to foreground the potential of the ethnographic approach to illuminate the more contested aspects of scaling, especially its ethics.

An ethnography of scalability has the potential to bring closer the worlds of academic discourses critical of development and practice-oriented fields concerned with pragmatic questions surrounding development interventions. By connecting the postcolonial critique of scalability and the recognition of a pragmatic need for scalability, such an approach can improve our understanding not only of what we can do to make scaling more effective but also of what the ethics of scaling means in a world fraught with inequality.

The relevance and urgency of these arguments is underscored by the unfavourable climate for ethnography both within and outside academia. After a recent conference at which I presented an empirical paper that discussed the findings of an ethnographic study, a recognised scholar in the field of international and comparative education approached me to ask a few questions about my work. During the conversation, it became clear that he assumed my paper was based on research I undertook during my PhD. Feeling self-conscious and wondering whether my work came across as somehow not up to the standards of ‘post-PhD academia’, I asked him how he knew. ‘It is obvious’, he told me bluntly. ‘No one but PhD students would have the time to do this type of ethnography these days’. His sobering comment is fully consistent with my experience of both academia and development practice: even though there are now perhaps more development experts than ever before who are trained in anthropology, long-term, sustained, immersive research is increasingly rare. The neoliberal pressures of ‘efficiency’, tight timelines, the bureaucratisation of academia, and an overemphasis on quantitative approaches and big data are just some of the factors that stack the deck against doing the kind of research for which I argue in this paper. It is precisely for this reason that we need to talk about ethnography.

Located in the ‘Global South’, the sites of scalability research in the context of educational development are not only sites of scaling but of social and economic deprivation that have, in formerly colonised parts of the world, experienced painful histories of exploitation and structural violence. In such contexts, an ethnography of scalability – not unlike the anthropology of development – is always inherently political and generative in nature. By giving voice to those actors who have been historically neglected and who function as effective translators of why scaling may or may not work, it helps to increase their agency in the scaling process and points to their potentially increased civic engagement. Such an approach is in line with Connell’s (2007) contention that the ‘South’ ought not to be seen merely as a site of extracting data to be theorised by agents in the ‘North’ but as a place where locally applicable theorisation of reality is evolved. An ethnography of scalability thus belongs to the genre of critical ethnography, whose aim is ‘to theorize social structural constraints and human agency, as well as the interrelationship between structure and agency in order to consider paths towards empowerment of the researched’ (Atkinson 2001, 193).¹¹ In other words, using ethnography to study scalability seeks to remove, as far as possible, the barrier between intervention and evaluation. Studying the impact of projects is often both costly and only indirectly linked to the goals of the intervention. Ethnography, on the other hand, has the potential to contribute directly to the goals of the programme (assuming the programme’s aims are aligned with the values of critical ethnography) while also generating important insights into the ethical dimension of scalability.

The idea that an ethnography of scalability inherently belongs to the genre of critical ethnography is a powerful one. It shifts the conversation away from the instrumental function of the voices of ‘development subjects’ often assigned to them by the technocrats of development and toward a recognition of the inherent value of the voices being heard – and the empowerment this might bring. Critical ethnography does not see informants merely as sources of data; rather, they are understood to be stakeholders who have agency in their communities’ social and political transformations. If conducted in an ethical manner, the process of research can become a vehicle for them to exercise this agency.

This latter point brings out a third layer of ethics in this debate. In addition to the ethics of intervention scaling (as represented by the third axis in Figure 1) and the ethics of research (closely linked to the notion of critical ethnography), the ethics of resource allocation is of crucial importance. In a world that lacks the willingness to commit sufficient resources to educational development, only some interventions can be scaled. We cannot afford to make bad scaling decisions. Collaborative in-depth qualitative research, such as multi-sited critical ethnographic studies of scalability, is arguably the best way to harness the collective knowledge of different stakeholders to ensure that we do the right thing.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Peter Sutoris is a PhD candidate and Gates Cambridge Scholar at the Faculty of Education at Cambridge University. He is the author of the monograph Visions of Development (New York: Oxford University Press, 2016) and director and producer of The Undiscovered Country, a film about education, development, and environmental degradation in the Marshall Islands. As a development practitioner, he has lived and worked in South Asia, the Pacific, the Balkans, and South Africa.

Notes

1. By ‘research’, I do not mean only studies constrained to academia but also applied studies undertaken by practitioner organisations and, indeed, collaborative research that bridges the two in significant ways.

2. This does not mean that ethnography is capable of fully capturing the value systems of individual stakeholders; rather, through its immersive, context-driven lens, it can facilitate a process of translation of the different meanings of ‘ethics’ shared by groups of people in different localities into the language of outsider practitioners and scholars, and thus help close the gap between those dismissive of the concept and those who see it as essential.

3. Extraction of natural and human resources, policies of de-industrialisation, and the boosting of the West’s self-perceived sense of racial superiority – eugenics models being at the core of some of these strains of imperialism – at the expense of suppressing indigenous cultures, languages, and ways of life were among some of the most ethically indefensible consequences of colonialism (Said 1979).

4. It is worth noting that interventions that contribute to the critical consciousness of learners are likely to be scaled up whenever scaling out occurs. This is because cultivating critical consciousness among a greater number of people often leads to the formation of social and political movements that are likely to influence policy and culture at large, leading to outcomes qualitatively different from those of the original intervention.

5. With notable exceptions (e.g. McCowan and Unterhalter 2015).

6. A parallel shift of focus can be observed among development practitioners, such as in the Problem Driven Iterative Adaptation approach developed by Andrews, Pritchett, and Woolcock (2013). They build a case for an institutional shift away from top-down approaches to development that seek to emulate ‘best practices’ in diverse conditions toward interventions based on positive deviance, experimentation, and built-in institutional learning mechanisms. In this approach, they argue, context plays a much greater role than it typically has in the ‘theory of change’ based on linear, emulative models of development historically espoused by large international development organisations set up under the Washington Consensus.

7. The charge of privileging breadth at the cost of depth is sometimes levelled against multi-sited ethnographies (Hage 2005). In considering this criticism, it is useful to consider that, while a multi-sited ethnography might lead to ‘experiencing a broader but possibly “shallower” world, […] understanding the shallow may itself be a form of depth’ (Falzon 2009, 9).

8. In one sense, the translation is what is under investigation; however, ethnographic approaches mean that many other levels and layers of data collection and analysis are possible.

9. The term ‘scaling object’ is used here in a broader sense than in the specific context of Education for Sustainable Development discussed in Mickelsson, Kronlid, and Lotz-Sisitka (2018), and it is seen as applicable to all development interventions.

10. Where the ‘threshold of acceptable ethics’ may lie is highly subjective. The goal of co-production within the context of a multi-sited ethnographic study into scaling is to try to identify patterns of convergence and divergence among different beneficiaries’ views with respect to where this threshold might lie for the larger community being targeted by the intervention.

11. An alternative definition of critical ethnography focuses on its being ‘structured in relation to our efforts to construct a mode of learning, and a conception of knowledge that may enhance the possibility of collectively constituted thought and action which seeks to transform the relations of power that constrict people’s lives’ (Simon and Dippo 1986, 196).