Instruments of lesson-drawing: comparing the knowledge brokerage of the OECD and the World Bank

ABSTRACT

This study examines the Organization for Economic Cooperation and Development (OECD) and the World Bank, the two largest global actors in the education sector, in their capacity as knowledge brokers. For these actors, positioning themselves between research and policy and circulating their versions of evidence has become a popular governance instrument to amplify their impact at the national level. To compare the strategies and targets of the OECD and the World Bank, we analyze three publication series: the OECD’s Education Policy Outlook and Reviews of National Policies for Education and the World Bank’s Systems Approach for Better Education reports. The results reveal significant differences between the OECD’s and the World Bank’s approaches to producing evidence and brokering knowledge. We interpret the differences against the backdrop of the idiosyncrasies of the two organizations: The World Bank sees itself as a transnational actor, and its knowledge production and brokerage are highly decontextualized transcending national experiences. By contrast, the OECD is an intergovernmental organization that views itself as a facilitator of cross-national peer exchange. Situating the findings within the broader framework of the global–national nexus, we argue that the World Bank’s approach promotes vertical policy learning, while the OECD’s approach pursues horizontal policy learning.

KEYWORDS:

• Policy learning
• international organizations
• evidence production
• knowledge brokering
• OECD
• World Bank

1. Introduction

Nowadays, intergovernmental organizations engage in global norm-setting by translating research evidence into national policy and practice. Unsurprisingly, this phenomenon of knowledge brokerage has been extensively investigated by researchers. In a single generation, researchers of global governance have moved from documenting unequal access to information to theorizing the impact of data surplus on national policy and planning. The new intellectual project aims to examine how global actors create evidence by means of data reduction. Scholars of global governance, for example, have assessed the popularity of international comparison in the domain of education in an ecosystem in which recourse to internationality or global connectedness enables the building of political coalitions and the mobilization of financial resources (Martens and Windzio 2022; Ydesen, 2019). In this study, we investigate whether there is a hierarchy of research evidence among different global actors and how these actors construct evidence for the purpose of lesson-drawing and knowledge brokerage.

Our study analyzes the two largest global actors in the education policy sector: the Organization for Economic Cooperation and Development (OECD) and the World Bank. We found significant differences in their technical approaches to producing evidence and brokering knowledge. The World Bank sees itself as a transnational actor whose knowledge production and brokerage are highly decontextualized, transcending national experiences. By contrast, the OECD considers itself to be an intergovernmental organization (IO) that facilitates policy learning and peer exchange among national policy experts to foster inter-country collaboration and peer learning. Keeping the global–national nexus in mind, we labelled the World Bank’s technical approach as vertical policy learning and the OECD’s approach as horizontal policy learning.

2. Re-visiting the normal distribution and the U-shape of educational reforms

Our study is inspired by two intriguing findings that Bromley et al. (2021) presented in their landmark study of educational reforms. They examined a total of 6,700 education reforms in 147 countries from 1960 to 2017 using policy-related publications from the OECD, the World Bank, and other knowledge-brokering organizations.¹ Reforms were defined as significant changes to the status quo as documented in the publications of the World Education Reform database. First, Bromley et al. demonstrated that in the 1991–2008 period, there was an explosive increase in neoliberal reform discussions worldwide. Since then, the reported reform activity has decreased significantly, coming almost to a standstill by 2017. In other words, when viewing educational reforms over a longer time period, one can observe that reform distribution peaked at the turn of the millennium. Second, Bromley et al. compared reform activity in terms of countries’ gross domestic product (GDP), finding that low-GDP and high-GDP countries issued more reforms than middle-GDP countries. In other words, they identified a U-shaped reform frequency: poor and rich countries scored high in terms of reform frequency, whereas middle-income countries scored low. Their findings constitute an important empirical contribution to punctuated equilibrium theories and research on reform cycles, indicating that periods of intense reform activity are followed by periods of little to no activity. The existence of reform cycles led Tyack and Cuban (1995) to distinguish between fundamental and incremental reforms. Using Tyack and Cuban’s distinction while drawing on Bromley et al.’s (2021) findings, one could argue that a period of significant reforms on a global scale ended around 2008. Since 2008, we have witnessed only minor or incremental reforms, whereby the neo-liberal reforms have been consolidated by scaling them “deep,” “out,” or “up” within a system (McLean and Gargani 2019). However, rather than assuming an apocalypse of reform enthusiasm, we wonder whether the sharp decline in recorded school reforms in the post-2007 era could be related to the thematic orientation of the school reforms compiled in the OECD, United Nations Educational, Scientific and Cultural Organization (UNESCO), and World Bank publications. In other words, would one notice the same decrease in reform frequency when drawing on school reform entries in the publications of other international organizations and then comparing them transnationally? Would one encounter multiple reforms instead of reform fatigue if one were to examine the publications of the International Lesbian, Gay, Bisexual, Trans and Intersex Association (ILGA) or Fridays for Future (FFF)? Furthermore, is the finding on the U-Shaped reform frequency rather a feature of the sample (the World Bank focusing mainly on poor countries and the OECD on rich countries) than general reform probability based on a country’s GDP?

These questions led us to think further: Previous research has shown that national policy actors prefer reports and grey literature over academic literature when justifying their policy decisions (e.g. James 2021; Steiner-Khamsi et al., 2020). Furthermore, if we acknowledge that IOs exert influence over education policy, among others, by means of policy recommendations, cross-national comparison, and, in the case of development banks, also by means of loans and grants (e.g. Littoz-Monnet 2017; Niemann and Martens 2018), the question becomes: do these two IOs differ in their choice of brokerage instruments? How do they broker knowledge and how do they steer national reforms in particular directions? In this paper, we seek to contribute to a better understanding of knowledge brokerage as an instrument of global governance. More specifically, our study investigates the differences between the OECD and the World Bank in how the organizations produce “evidence” for national policymakers and broker between research and policy, as well as what countries the two global actors cover in their policy-related reports and studies.

3. The rise of knowledge brokers in an era of information surplus

A few words on the transformation of global actors into knowledge organizations and, more recently, into knowledge brokers are in order.

The metamorphosis of the World Bank from a financial institution with a global reach to a knowledge bank that lends both money and ideas was first observed by Jones (2004) and Stone (2000) and then further documented by others (De Francesco and Guaschino 2020; Steiner-Khamsi 2009; Zapp, 2017, 2021). According to the archival research of Jones (2004), the concept of an international knowledge bank was first discussed by the Board of Governors of the World Bank in March 1996. One of the issues discussed was whether the financial lending operations should be delegated to the regional development banks (e.g. the Asian Development Bank and the African Development Bank) while the World Bank itself would focus on lending ideas. Three years later, in 1999, the World Bank’s Global Development Network was launched at a conference in Bonn (see Stone, 2000). The idea was to treat local best practices as “public goods” and make them globally available. As a result, policy transfer would ideally occur within and among the countries of the Global South, replacing the previous practice of transplanting reform packages from the Global North to the Global South. Although the World Bank has not decreased its role as a money bank, over the past decade, it has increasingly started acting as a global monitor and lender of “best practices.”

Similarly, the OECD’s ability to exert soft power by means of international comparison and global norm-setting has been extensively studied by Kerstin Martens and her colleagues in the education policy domain (Martens and Jakobi 2010; Niemann and Martens 2018) and by other social researchers in other policy domains (for example, Littoz-Monnet 2017). The OECD’s practice of knowledge brokering has been studied by Seitzer (2022), who found that although some countries are unsurprisingly at the centre of the international organizations’ (IOs’) knowledge brokering efforts—namely, the European “poster children” such as France, Germany, and Finland—the OECD still places great importance on regional similarities. In some regions, mostly Eastern Europe and Spanish-speaking countries, knowledge was traded regionally rather than with the “poster children”.

The World Bank and the OECD are not alone in first producing international comparative studies based on numbers and rankings, then offering their own interpretations of the findings to generate problem awareness, and finally presenting a global portfolio of interventions as “best practices” for solving problems or policy issues at the national level. The dual process of decontextualizing information by means of numbers and then recontextualizing the numbers using one’s own narratives (Espeland 2015) is key to the attraction of governing by numbers for policy. Similarly, global actors’ practice of both diagnosing a problem and prescribing a remedy has been strongly criticized for quite some time (Samoff 1999). This dual process of de- and recontextualizing by means of indicators as well as the overlap between policy analysis and policy imposition are controversial but not unusual practices for IOs. However, the World Bank and the OECD are undoubtedly the most influential multilateral organizations when it comes to national education systems. For this reason, our study focuses on how these two global actors bridge research and policy vis-à-vis national governments.

Both global actors actively engage in knowledge brokerage by bridging knowledge producers (policy researchers) and knowledge users (policy makers). Their brokerage involves establishing globally binding indicators, producing policy-relevant studies, disseminating research syntheses, preparing evidence-based recommendations, reviewing good practices, and other activities meant to strengthen the connection between science and politics (see, for example, Grek and Ydesen 2021). Naturally, each organization does this within its own knowledge production paradigm: the World Bank aims to increase prosperity, and the OECD aims to address labour market needs (Seitzer 2021).

Knowledge brokerage per se is not a new phenomenon. In fact, the study of knowledge brokerage dates back to Caplan’s (1979) two-communities theory. As Cairney (2016) and MacKillop, Quarmby, and Downe (2020) astutely pointed out, the polarization between science and politics sometimes suggests a distinction between “truth” or “facts” (produced by science) and action or decisions (taken by politics). Individuals and organizations dedicated to transferring, translating, or “pushing” research, truth, and facts onto political decision-makers (e.g. bridging the two communities) are referred to as brokers, intermediaries, or boundary spanners. However, the two-communities theory requires a more complex explanation because the two spheres are often structurally coupled, generating an area of overlap in which “research evidence” for policy and planning is produced by both communities (Steiner-Khamsi et al., 2020). This structural coupling between science and politics is not without its challenges. As Eyal (2019) and Maasen and Weingart (2005) have convincingly argued, the proliferation of evidence-based policy planning has paradoxically fuelled the crisis of expertise. In fact, not only has science become politicized and politics scienticized, but science has also become demystified in the eyes of the public:

[…] the very recourse to expertise increases uncertainty and threatens legitimacy because now the public is witness to controversies between scientists […]. (Eyal 2019, 102)

This crisis particularly affects the actors and institutions of regulatory science. Eyal (2019) identified three communities (or “lanes”), with regulatory science (or policy studies) in the middle lane, situated between “pure science” and “regulation.” Clearly, the rapid advance of associations, think tanks, nongovernmental organizations, and government-sponsored institutions that produce, spread, and broker policy knowledge is remarkable (Stone et al., 2020). Naturally, policy brokers have also found their place in theories of policy advisory systems, regulating the flow of information between science and policy (Howlett 2019). In recent years, this advance has triggered an avalanche of publications on knowledge brokerage. In their systematic review of the literature, MacKillop, Quarmby, and Downe (2020, 399) asserted the following: “These multiple definitions are symptomatic of the growth of the field and of various researchers and disciplines’ attempt at making sense of and implementing/refining these new processes” (399). The rise of knowledge brokerage is not surprising given the limited uptake of global public goods in the form of freely accessible databanks, toolkits, studies, repositories of case studies, and best practices.

However, addressing the US context, Lubienski (2019) claimed that there is not a scarcity but a “surplus of evidence.” In such a “marketplace of ideas,” there is ample opportunity for new, nonstate actors, specifically the private sector, to serve as intermediaries between research production and policy making:

Into the chasm between research production and policymaking, we are seeing the entrance of new actors—networks of intermediaries—that seek to collect, interpret, package, and promote evidence for policymakers to use in forming their decisions. (Lubienski 2019, 70)

However, the surplus of evidence also increases the risk of misinformation being mixed into trustworthy policy advice (Perl, Howlett, and Ramesh 2018). In fact, the survival of the OECD and World Bank as leading knowledge brokers in education rests on the success of these IOs to defend their status as leading experts on education policy advice. The proliferation of knowledge repositories and databanks as well as the limited uptake of national policy and planning are therefore issues of great concern for global actors. The World Bank’s Systems Approach for Better Education Results (SABER) initiative is a case in point. Until fall of 2021, SABER’s download statistics were publicly accessible. A cursory search of these statistics revealed that what is considered the World Bank’s flagship education initiative had only received 1,108 downloads² of its overview text³ (Rogers and Demas 2013) between January 1, 2014, and October 22, 2021. Over half of the downloaders (n = 574) were based in the US. Thus, on average, one person downloaded a text from the SABER platform every three weeks. This is very little given the enormous human and financial resources invested in developing this platform.⁴

Given the contradiction between knowledge surplus and uptake scarcity, this study investigates how the two largest intergovernmental organizations—the OECD and the World Bank—bridge research and policy as knowledge brokers and which countries constitute their data source as well as target audiences.

4. Research design and database

This section outlines the methodological and sampling rationale behind our examination of the OECD’s and the World Bank’s influence in promoting certain policies among their member states.

As discussed before, the OECD and the World Bank are leading drivers in the production of policy-relevant information, data, benchmarks, and reports. However, the two IOs have vastly different agendas and means at their disposal. The OECD’s agenda states that the IO “helps individuals and nations to identify and develop the knowledge and skills that drive better jobs and better lives, generate prosperity and promote social inclusion.”⁵. Meanwhile, the World Bank states that “ensuring [that] learning happens – for everyone, everywhere”⁶ is its ultimate goal. Whereas the OECD offers policy evaluation, the World Bank provides technical assistance, loans, and grants for systemic reforms to all stages of education. The World Bank supports the diffusion of its ideas with its lending power, while the OECD has to rely on the persuasiveness of its publications and data. Thus, the two IOs have different means of brokering information, policies, and data, which indicates that they may have different approaches to “bringing evidence” to policymakers. These differences in available means, combined with the unique status of the two IOs as leading knowledge brokers in education policy, make the comparison of these two cases relevant.

Both IOs are highly active in publishing policy reports and studies that evaluate existing policies and recommend reforms. These publications can either take the form of country-specific reports requested by member states or can be part of the IO’s service portfolio, which provides information and global comparisons. We focused on three publication series in particular aimed at monitoring and improving policies across all levels of education: the OECD’s Education Policy Outlook (EPO) and Reviews of National Policies for Education (RNPE) and the World Bank’s SABER reports. We excluded publication series that address issues related to a specific level of education (e.g. the OECD’s Higher Education series). Instead, we focused on broader series, following the sampling approach of Bromley et al. (2021), which inspired this study, as mentioned above. However, in contrast to Bromley et al., we did not include UNESCO’s World Data on Education series because it focuses on the structure and legal framework of education and does not include policy recommendations and comparisons as do the other publications in our sample. As we examine both coverage and brokerage—that is, policy recommendations that potentially lead to reforms—the World Data on Education series would not have added substantial information to our analysis.

4.1. Country sample

Our sample included 131 reports from the OECD and 145 reports from the World Bank (see Table 1). The publications were published between 1998 and 2021 in varying intervals and collected in early 2022. It is important to note here that the sample of countries that received these reports was not limited to the member states. Both IOs (particularly the OECD) also prepared reports for countries that were not members.

Table 1. Sampled documents.

IO	Reviews of National Policies for Education	Education Policy Outlook	Systems Approach for Better Education
	1998–2020	2015–2019	2009–2021
OECD	60	71
World Bank			145

All examined report series were intended to serve not only as a monitoring device but to also include policy recommendations for reforms. The EPO series described itself as “an analytical observatory that monitors the evolution of policy priorities and policy developments from early childhood education to adult education, mainly among OECD education systems, to provide a comparative understanding of how policies are evolving, and how they can be best implemented or improved over time.”⁷ In this series, each publication is prefaced by a framing report, which is followed by shorter country-specific reports on the later pages. These shorter country-specific reports were extracted and used for analysis. The RNPE series described itself as conducting “analyses of education policy development and implementation in response to or anticipation of wider economic and social trends and developments” to provide countries with a “range of national policy options and strategies.”⁸ Whereas the EPO reports were published by the Directorate for Education and Skills, the production of the RNPE series involved ministries and professional groups or researchers.

The World Bank’s SABER series focuses on the production of comparative data “to conduct a thorough inventory of [countries’] education policies and institutions based on global best practices, as well as provides decision makers and stakeholders at all levels with tools for structured and effective policy dialogue.”⁹ The data for these specific publications is collected by the World Bank using a questionnaire provided to local experts and validated by the World Bank via discussions with government counterparts. In addition, the SABER series compares country results with a global benchmark regarding policy effectiveness.

To summarize, whereas other publication series by the two IOs function only as monitoring devices, the series examined in our study included in their mission statements the goal of providing policymakers with concrete recommendations for reforms. Such recommendations can potentially lead to policy learning and diffusion because they present certain policies as the “gold standard” or “best practice,” buzzwords that every country wants to hear about their own policies.

4.2. Analysis

To compare the strategies and targets of the OECD and the World Bank in bringing evidence to policymakers, we examined the aforementioned country reports. More specifically, we first looked at the number of reports prepared for specific countries to investigate which countries were rigorously monitored by the IOs and which were not monitored at all. These countries were designated as targets of policy advice. Second, we investigated the origins of policy recommendations by analyzing the frequency with which countries were mentioned in the reports and used as reference societies for policymakers. This allowed us to identify policy darlings—that is, countries whose reforms were repeatedly documented or referenced in other country reports. Similarly, we also identified policy orphans—that is, countries whose reforms were not reported or referenced for lesson-drawing in other country reports.

In our analysis, we used the automatic text analysis of the software R to extract country references, specifically country names and demonyms (while ignoring tables and figures), from the reports. Due to the nature of automatic text recognition, some references may have been overestimated because the software did not recognize tables. However, the highly frequent mentions of certain countries remain a significant result. These references to different countries were often linked to specific policy recommendations. For example, the RNPE report on Chile for 2004 stated that “the Ministry has encouraged a dual model, adapting experience from Germany to Chilean conditions” (OECD 2004, 81). In this instance, Germany was used as the reference society for Chile by introducing Chilean policymakers to a German policy on the dual education system for secondary-professional and technical education. In theory, these references can also be negative; a negative reference could serve as a deterrent to implement specific policies. However, in this study, we did not further differentiate the nature of the references because even if a policy highlighted by either IO serves as a negative example, it still results in policy learning and can be interpreted as an example of lesson-drawing. Furthermore, it is highly unlikely for an IO reliant on their membership dues such as the OECD or the World Bank in which member states are literal stockholders, to refer to their members and partners as negative examples in their reports. If critiques are made, they are expressed in the form of country evaluations instead and would not appear in our sample as a reference.

Finally, we investigated whether the reference patterns depended on several country-level predictors: First, we recorded if a country had received a report from the OECD or the World Bank in the respective year to highlight if countries not receiving reports are also used as poster children. Then, we included the V-Dem index of Electoral Democracy (Coppedge et al. 2023) which has been identified as an important factor for the use of evidence-based policymaking (Maasen and Weingart 2005). We also controlled for the income group based on the World Bank classifications, to shed further light on the correlation between income and being the focus of the World Bank or the OECD. We also tested for the influence of the number of reforms as reported by Bromley et al. (2021). Finally, we included tracking as a measure of the level of education system standardization and enrolment in primary and post-secondary schooling as measures for equality of access as specified by West and Nikolai (2013), and the participation in PISA as a measure of compliance with OECD demands, as well as time (nonlinear) and a random intercept for each country.

5. Evidence production and dissemination: a comparison

In this section, we first identify the countries that were reported on by the OECD and the World Bank and discuss the impact of income and region on report coverage. Furthermore, we examine the countries referenced in these reports to identify which countries received policy recommendations from which organization. Our analysis revealed that the OECD and the World Bank differed in their approaches to evidence production and dissemination of policy knowledge.

5.1. The number of countries covered by the OECD and the World Bank

Overall, the World Bank provided greater coverage in its reports and had a stronger focus on developing countries than the OECD. The World Bank published reports on 86 countries, while the OECD published reports on 56 countries. Among these countries, only 19 were covered by both the World Bank and the OECD. Out of 195 countries that the United Nations (UN) currently recognizes (193 member states and two permanent observer states), 119 countries were reported on by either or both IOs.¹⁰ Consequently, the educational systems of 70 countries were left unexplored or un-reviewed. Interestingly, in line with Bromley et al.’s (2021) findings, we found that high- and low-income countries were more likely to be covered by the reports than middle-income countries (p < .05). Being a high- or low-income country increased the log odds of coverage by 0.66. Furthermore, our analysis showed that a country’s Gross national income (GNI) per capita was statistically significantly related to which IO covered the country (see Table 2). We found that compared to high-income countries, upper- and lower-middle-income countries were more likely to be covered by only the OECD instead of the World Bank or both.

Table 2. Multinomial logistic regression model of the IO coverage of countries.

	Covered by:
	The OECD only vs. both	The OECD only vs. World Bank only
Income (Ref. high-income)
Low-income	−16.182 (1050.431)	−18.330 (1050.431)
Lower-middle income	−3.367 (1.304)***	−4.366 (1.109)***
Upper-middle income	−2.962 (0.786)***	−2.625 (0.628)***
Constant	2.269 (0.606)***	1.421 (0.421)***
Pseudo R	0.30
Observations	121
Log-likelihood	−82.951

Note: *p < 0.1; **p < .05; ***p < .01

When looking at IO coverage in relation to a country’s regional affiliation, we found a statistically significant relationship (χ² = 67.54, p < .001). For example, almost 94% of African countries were covered only by the World Bank, compared to only 2.7% of European countries. A large percentage of Asian countries (72.73%) were also covered only by the World Bank. Most European countries were either covered only by the OECD (67.57%) or covered by both the OECD and the World Bank (29.73%) (Table 3).

Table 3. IO coverage of countries by regional affiliation in percentage.

Type	Africa	Americas	Asia	Europe	Oceania	Total
OECD	3.12	35.71	12.12	67.57	28.57	30.08
World Bank	93.75	50.00	72.73	2.70	71.43	54.47
Both	3.13	14.29	15.15	29.73	0	15.45

Note: χ² = 67.54, p < .001

The findings presented in this section may not be too surprising when one considers the OECD’s membership composition, which includes mainly the rich and/or Western countries, as well as the World Bank’s focus on development. However, the findings have important implications for researchers and policy actors interested in examining the reports produced by these IOs in terms of understanding “global” and “inter-national” trends. The IOs’ report coverage reflects not only which countries receive attention from the IOs but also which countries are left unexamined. This indicates that the knowledge produced and transferred by different IOs may overrepresent or underrepresent knowledge from particular regions and countries.

5.2. Who are the reference societies for the OECD and the World Bank?

The previous section explored which countries are covered by the OECD and the World Bank reports, showing that the World Bank had a wider reach than the OECD and that particular countries were “orphaned” and not reported on at all. Given the different report coverage of the two IOs and their uneven focus across countries, it is important to ask the following questions: Which countries are used as reference societies in these reports? And: do the two IOs differ in how their country reports reference foreign countries? Answering these questions will allow us to show how the strategies of these IOs in providing knowledge for policy vary and highlight the targets and origins of these reform suggestions.

Table 4 shows descriptive statistics on the distribution of references in each of the organizations’ reports. It becomes clear that the OECD makes greater use of references, as the average number of references per report as well as the average number of countries referenced per report are greater in OECD reports than in World Bank reports. When looking closely at the countries referenced in the reports of the OECD and the World Bank, we find an astonishing number of country references in all the reports. Across both IOs, a total of 28,925 references to 167 different countries were made. Although some countries were mentioned in multiple reports several times, a large number of countries were mentioned only once or in very few reports. The majority of references were made by OECD reports, specifically by the RNPE series. Interestingly, not only did the OECD reports contain a larger number of references to different countries in each report, but these reports also referenced a greater variety of countries overall than the World Bank’s reports. The OECD’s reference coverage spanned 167 countries, while the World Bank referenced 143 countries. All countries referenced by the World Bank were also mentioned by the OECD; however, 24 countries were mentioned only by the OECD. Consequently, of all the UN-recognized countries, 28 were not mentioned by either of the IOs and could be considered “orphans.” The policies of these countries were not taken up as exemplars for emulation or deterrence. This dynamic, where the OECD reports are referencing more countries frequently, and the World Bank are referencing fewer countries sparingly is contrary to the previous finding, whereby the World Bank covered more countries in its reports than the OECD. These differences indicate that the two IOs diverged in their knowledge-brokering strategies. The World Bank produced reports on a greater number of countries, while the OECD produced reports on fewer countries but drew lessons from a larger pool of exemplars.

Table 4. Descriptive statistics on the references in the OECD and World Bank reports.

IO	N of references	N of the countries referenced	Avg. N of references per report	Avg. N of countries referenced per report
OECD	23008	167	292	52.9
World Bank	5917	143	37.5	16

When reviewing the frequency with which both IOs referenced countries over time, we followed the assumption that these references ultimately led to reforms or policy changes. In line with Bromley et al.’s (2021) findings, we identified a reference hike shortly before 2005, when the reform numbers were highest. Afterwards, this reference trend slowed down. Figure 1 shows the cumulative count of references per year divided by the IO, along with the number of reforms as collected by Bromley and colleagues in the respective time frame. The difference in reference frequency by IO was significant (t(2190.2) = 16.649, p < .001) during the relevant time period (2009–2021) when both organizations published reports. We see that the OECD made country references more frequently than the World Bank. The reference frequency peaked around the year 2016/2017. In addition, the number of education reforms plateaued around 2005, much earlier than the number of references did. This is an interesting result that, on the one hand, allows us to question the effectiveness of the reports and IO knowledge brokering efforts and that, on the other hand, may predict another reform wave in the near future, if we follow the assumptions of equilibrium theories (Tyack & Cuban, 1995).

Figure 1. Reference frequency over time with reform frequency.

Another interesting result was that the number of countries mentioned in each report series differed: the EPO series by the OECD referenced only 42 countries, and the focus of this series was on reporting rather than reforming. By contrast, the RNPE series by the OECD mentioned 167 countries, and the World Bank’s SABER series mentioned 143 countries. That said, the difference between the two IOs regarding the average number of countries referenced was statistically significant, t(139.57) = 3.91, p < .001, even when excluding the RNPE series. This shows that even in reports focused on the status quo rather than reforming the system, as was the case in the EPO series, the OECD still made more references to different countries than the World Bank. Therefore, although the number of countries referenced depended on the series, the difference in the number of countries mentioned, as well as the overall number of references made, was significantly different between the two IOs.

The two IOs also differed regarding which countries they referenced more frequently. Figure 2 shows the reference coverage and frequency of both IOs side by side. The US was the most referenced country in the OECD reports with a total of 2,057 references, followed by England with 906 references, and then the UK with 804 references. The subsequent most frequently mentioned countries were Australia (786), Germany (723), and only then Finland (713), the celebrated winner in the first round of the OECD’s programme for international student assessment (PISA). The most cited country in the World Bank reports was the US (1217), followed by the much less referenced Singapore (336), Japan (247), England (238), Korea (235), and China (197). The difference in the two IOs’ policy darlings further confirms their regional affiliations and the difference in the overall number of references may reflect their unique approaches in lesson-drawing: the OECD more frequently references other countries to provide examples of policy recommendations, while the World Bank rather refers to its own standards than referencing different countries.

Figure 2. Reference coverage of the OECD and the World Bank. Note. The dotted countries were not referenced at all by either IO.

To further examine which factors are related to countries’ chances of being used as reference countries, we fitted multinomial regression models which test for the chances of being referenced by (1) both IOs, (2) the OECD, or (3) the World Bank, in contrast to not being references at all in the respective year. An additional model (model 4) tests the chances of being referenced by the OECD in contrast to being referenced by the World Bank (see Table A1 for all models).

Having a report from the OECD increased the log odds of being referenced by the OECD or both IOs. Having a report from the World Bank, however, increased the log odds of being referenced only by the World Bank. When contrasting to the references from the OECD reports, the effect was even negative (−0.68, p < 0.00). This suggests that both IOs more frequently referenced the countries that they each produced a report on, and the OECD less frequently referenced countries that the World Bank produced reports on. Electoral Democracy had a positive significant coefficient in all three models, suggesting that democratic countries were more often used as reference countries. In contrast to high-income countries, the log odds of being referenced by the OECD were significantly smaller for all other income groups. The OECD appears to have a preference for referencing countries with a higher income. In contrast, the log odds of being referenced by the World Bank were greater for lower-middle and low-income countries than for high-income countries. This supports our previous assumption that the World Bank focuses on low- and lower-middle-income countries, while the OECD focuses on high-income countries. Figure 3 shows the marginal effects of the income groups with a 95% confidence interval, comparing the probability of (1) being referenced, (2) being referenced by the OECD, (3) by both IOs and (4) by the World Bank respectively. The figure shows that low- and lower-middle-income countries have the greatest probability of not being referenced at all. High-income countries have the greatest probability of being referenced by the OECD, while low-income countries have a greater probability of being referenced by the World Bank than the countries in other income groups.

Figure 3. Predicted probabilities of references in IO reports.

Interestingly, the number of reforms had no significant effect. PISA participation however had a small but positive effect for both IOs. Both primary and post-secondary enrolments increased the odds of being referenced by the OECD as well as both IOs; however, only post-secondary enrolment influenced the log odds of being referenced by the World Bank.

These results demonstrate not only the geographical focus of the two IOs but also their different approaches to policy knowledge brokering: while the OECD takes a horizontal, peer-learning approach, the World Bank takes a vertical approach. Both the OECD and the World Bank seemingly employed the practice of lesson-drawing in their country reports; however, the OECD more frequently referred to the countries that they produced country reports on as examples of policy learning. This finding corroborates our previous descriptive finding that although the OECD writes reports on a limited number of countries, the OECD are likely to use them as policy learning examples in other countries’ reports. By contrast, the World Bank writes reports on a significantly higher number of countries but references them much less. The question remains then, from where the World Bank draws knowledge for lesson-drawing in their reports? According to the World Bank’s “Learning for All”-Report (2011) which describes the strategies for SABER, the World Bank has developed “systems tools” that ensure cross-country comparability and allow countries to evaluate their education systems against the benchmarks of the SABER-tool. These benchmarks, developed with experts and policymakers from all over the world, are supposed to support countries in finding optimal policies for governance institutions, accountability, information, financing rules, and school management. The World Bank’s self-referencing pattern suggests however that while the World Bank might draw on the information on countries’ education system to develop its benchmarks, those education systems are only measured against education system characteristics within their SABER-tool, rather than being directly compared to other countries’ education systems.

6. Conclusions

Over the past three decades, several global actors have expanded their reach as knowledge banks to include both knowledge production and knowledge brokerage. Arguably, the rise of knowledge brokerage has been fuelled by the dictum of evidence-based policy decisions in an era of information glut and a surplus of “evidence.” In this study, we focused on the two most significant actors among the international knowledge brokers in the education policy sector: the OECD and the World Bank. According to Jarl Bengtsson, a former head and long-time staff member (1971–2002) of the OECD’s Centre for Educational Research and Innovation, the 1980s were the time when the centre started to develop its “strong focus on educational standards and interest in indicators” (Bengtsson 2008, 2), which led to its International Education Indicators (INES) programme, the Education at a Glance series, and finally PISA (see also Grek and Ydesen 2021). The World Bank also started to invest in indicator development in the late 1980s. The organization progressively expanded its databank from 116 indicators in April 1989 to 1,600 indicators in October 2018.¹¹ The path to knowledge brokerage was similar for both IOs. Each established a quantification apparatus that produced data-based policy analyses and disseminated their recommendations in the form of research syntheses. In this regard, they are not impartial knowledge brokers but act as policy brokers that transfer their own portfolios of good practices or policies from the global to the national level (Stone et al., 2020). Two decades later, the other three large IOs in education followed suit and established themselves as knowledge brokers: UNESCO established its Institute for Statistics in 1999 and the Global Monitoring Report unit in 2002; the multi-stakeholder IO Global Partnership for Education was only established in 2003 (under the name of Education for All-Fast Track Initiative); and finally, UNICEF started to develop its quantification regime in the new millennium (Cairney 2016). The OECD and the World Bank can therefore be considered norm setters not only for education policymaking through their policy recommendations and brokerage but also for competing IOs. Even though the OECD and the World Bank are not the only knowledge brokering IOs in education policy, they are certainly the most visible actors to engage in knowledge brokering through numbers and expertise.

In this study, we demonstrate that it is important to consider the information sources (i.e. the OECD and the World Bank), which significantly impact the interpretation of the findings. We offer further insights into the association between GDP per capita and reform discussion activity observed by Bromley et al. (2021) that this association might be dependent on the data source: OECD and World Bank reports. For example, reforms in high-GDP European countries were mostly reported on by the OECD (67.57%), whereas reforms in low-GDP African countries were almost exclusively (93.75%) discussed in reports produced by the World Bank. It suggests that countries not covered by the reports, the so-called “orphans”, might still be very active in education reforms. Rather than interpreting that the unmentioned countries in our sample did not have any reform activity, the findings suggest that a large number of countries are neither of interest to the OECD nor to the World Bank. In a similar vein, we detected that especially middle-income countries were neglected by both the OECD and the World Bank as they were often not OECD members and not IDA-credit eligible.

As discussed by many other scholars, the OECD and the World Bank differ in terms of their mandates (Bromley et al. 2021; Niemann and Martens 2021; Ydesen et al., 2022). In the education sector, the OECD’s mandate focuses on developing skills for the 21st-century labour market, whereas the World Bank legitimizes its education portfolio within its own institution by framing education as human capital that benefits future productivity and economic growth. Nevertheless, both organizations have been the first drivers of neoliberal reforms by advocating, for the past three decades, a reduction in the role of the state in standard setting and monitoring (including monitoring students’ learning outcomes by standardized tests) and an incentivization of the private sector’s involvement in the public sector (including running private schools with funding from public resources). Moreover, these IOs do not simply bridge the realms of science and politics by means of knowledge brokerage; rather, they act as policy brokers that report on specific types of school reforms based on what they consider to be effective education policies (Bali, Capano, and Ramesh 2019). This means that the interpretation and translation of different bodies of knowledge that these IOs offer to national policy actors are significantly shaped by their institutional and ideational dimensions of policy brokerage (Béland 2016; Thelen, 2004). Therefore, it is not surprising that the World Bank and the OECD have different approaches to knowledge brokering and consequently also different target audiences.

In particular, in this study, we found institutional differences in the choice of policy brokerage instruments. The OECD makes more references to a wider variety of countries and does so much more often than the World Bank does. This result reflects the two IOs’ unique approaches to knowledge brokerage. The recent literature on policy instruments (Baek 2022; Verger et al., 2019; Ydesen et al., 2022), procedural policy tools, and “new” design orientation in policy sciences (Bali, Capano, and Ramesh 2019) departs from the problem-solving assumption commonly associated with policy analysis. Simons and Voß (2018, 15) provided a succinct account of how the field of policy instrument analysis has changed over time from “instruments as tools to instruments as institutions to instruments as webs of practices” (15). We found it quite striking that the two IOs conceptualize and operationalize knowledge brokerage differently. The World Bank makes far fewer references to other countries; instead, it refers to its own gold standard of education. SABER includes recommendations for an education model developed by the World Bank and recommended to all countries, regardless of their location, cultural heritage, and economic situation. This one-size-fits-all solution developed as an amalgam of education policies from around the globe, is a truly global model. Therefore, the World Bank maintains a universal or transnational approach to policy learning. It seeks to identify the single best solution that is then disseminated and funded in its target countries. In sharp contrast, the OECD’s rationale for policy brokerage is different. In the education policy domain, the OECD functions very much like a think tank that uses international comparisons to promote skills for the 21st-century labour market. The OECD’s scoring and ranking of educational systems produce narratives (Espeland 2015) that are generated at the headquarters with substantial input from national experts. In PISA, the OECD’s close collaboration with the test administrators is indispensable at all stages of the test, including interpreting the results of PISA. The PISA policy instrument requires collaboration to interpret the decontextualized content of the test. It is perhaps the features of this particular policy instrument that explain why the OECD focuses on peer learning through its partnership programme and encourages countries to closely work together on solving shared problems. This approach is an international one, as the focus is on interactions between countries and peer learning rather than finding a common solution for all countries.

These two different approaches to policy brokerage—the OECD’s international peer learning approach versus the World Bank’s transnational best-solution approach—are embedded in different constituencies (see Béland and Howlett 2016; Mukherjee and Howlett 2015; Simons and Voß 2018). The OECD’s contextual approach requires national experts as research allies, whereas the World Bank’s universalist approach resonates with development economists who administer randomized controlled trials to find a one-size-fits-all solution for educational policy challenges. The OECD’s horizontal policy learning (encouraging learning from peers and collaborating with others) and the World Bank’s vertical policy learning (encouraging learning from a portfolio of “good practices” developed or identified by the World Bank) are situated in different institutional ecosystems. The OECD’s partners are sector researchers at the country level, whereas the partners of the World Bank’s education experts are the economists within the bank. It is interesting that the World Bank, despite recognizing the greater effectiveness of a more collaborative approach to policy brokerage (Bazbauers 2020), still relies on its vertical policy learning approach. This finding raises the question, if the OECD’s horizontal approach to policy brokerage gives the IO its competitive edge in the future, and whether or not the World Bank would also adopt a similar approach.

To better understand the knowledge brokerage instruments produced by IOs, we call for further research in two areas. First, we suggest taking a closer look at countries not included in the country reports by either of the two studied IOs. What are the features of the orphans? Second, a follow-up qualitative analysis of country references would enable us to understand why exactly the US, England, Australia, and Germany are mentioned the most by the OECD and why the US, Singapore, and Japan are mentioned the most by the World Bank. In line with previous comparative studies, it would be helpful to know to what extent references to specific educational systems are, in fact, projections that reflect an institution’s own ideational stance on certain education policies (Steiner-Khamsi & Waldow, 2018). We also suggest investigating knowledge brokers beyond the “usual suspects” such as the OECD and the World Bank to identify additional policy brokering approaches. Finally, our comparison of the varied policy brokerage instruments applied by the two IOs focused on the policy domain of education. A cross-sector comparison would enable us to assess whether an IO’s choice of instrument is domain-specific or institutional.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Deutsche Forschungsgemeinschaft [grant number 374666841-SFB 1342].

Notes

1 Available at https://www.werd.world/database.

2 Statistics at: https://documents.worldbank.org/en/publication/documents-reports/downloadstats?docid=867151468180272110

3 See also https://documents.worldbank.org/en/publication/documents-reports/documentdetail/867151468180272110/the-what-why-and-how-of-the-systems-approach-for-better-education-results-saber

4 However, significant variation exists according to topic and publication type. To assess the downloads by publication type, in October 2021, we examined five randomly selected publications for each publication type; every fifth publication was selected for analysis. With the exception of a few country reports, such as SABER Early Childhood Development Iraq Country Report (in English) published in 2013, the SABER country reports are not popular, ranging between 175 and 358 total downloads. The SABER topics series is consulted for some topics more often than for others. Users seem to consult the platform for topics that are closely associated with the World Bank’s area of expertise—namely, an economic perspective on education, as in the case of the popular SABER workforce development report (in English) published in 2017. A notable exception that is possibly diametrically opposed to other popular topics on the SABER platform is the article on social-emotional learning published in 2014 entitled “Learning and Resilience: The Crucial Role of Social and Emotional Well-Being in Contexts of Adversity” (English), which has reached 2,849 downloads. Limited uptake in an era in which there is a proliferation of digital platforms for education policy has clearly become an issue, especially for non-networked organizations. Note that the publicly accessible download statistics of the SABER site have been deactivated since December 2021.

5 Available at https://www.oecd.org/education/

6 Available at https://www.worldbank.org/en/topic/education.

7 See OECD website: https://www.oecd.org/education/policy-outlook/.

8 See OECD website: https://www.oecd.org/education/school/reviewsofnationalpoliciesforeducation.htm.

9 Further explanation see: https://www.worldbank.org/en/topic/education/brief/systems-approach-for-better-education-results-saber.

10 In addition, the World Bank report coverage includes Zanzibar, Kosovo, and the Palestinian Territories. The OECD reports cover the UK, England, and Scotland separately.

11 See https://datatopics.worldbank.org/world-development-indicators/stories/world-development-indicators-the-story.html.

Appendix

Table A1. Multinomial regression model of policy references in IO reports.

	References from both IOs vs. no references				References from the OECD vs. no references				References from the World Bank vs. no references				References from the OECD vs. the World Bank
	Model 1				Model 2				Model 3				Model 4
Predictors	Coef.		Std. Err.	P > z	Coef.		Std. Err.	P > z	Coef.		Std. Err.	P > z	Coef.		Std. Err.	P < z
Report from OECD	1.418	***	(0.46)	0.000	1.038	***	(0.35)	0.004	1.199		(0.73)	0.105	−0.004		(0.62)	0.230
Report from the World Bank	0.222		(0.45)	0.625	−0.157		(0.41)	0.707	0.987	***	(0.33)	0.004	−0.685	***	(0.25)	0.000
Electoral Democracy	3.978	***	(0.96)	0.000	2.072	***	(0.52)	0.000	1.497	***	(0.57)	0.009	1.024		(0.41)	0.114
High income	Reference		–	–	–		–	–	–		–	–	Reference	–	–	–
Upper middle income	−.166		(0.55)	0.763	−0.790	***	(0.30)	0.009	0.499		(0.35)	0.164	−0.430	**	(0.36)	0.040
Lower middle income	−0.598		(0.70)	0.396	−1.293	***	(0.36)	0.000	0.806	*	(0.42)	0.059	−.835	*	(0.29)	0.054
Low income	−.795	*	(1.06)	0.092	−.238	**	(0.49)	0.012	1.177	**	(0.52)	0.025	−.965	***	(0.33)	0.003
N of reforms	0.07		(0.04)	0.14	0.016		(0.02)	0.573	−.069		(0.06)	0.287	−.034		(0.02)	0.101
Tracking	−.691		(0.52)	0.184	−.132		(0.29)	0.655	0.461		(0.29)	0.118	−.095		(0.18)	0.607
PISA	1.227	***	(0.39)	0.002	0.381	**	(0.18)	0.039	0.516	**	(0.28)	0.067	0.481	***	(0.15)	0.002
Primary enrolment	0.016	*	(0.00)	0.077	0.015	***	(0.00)	0.003	0.006		(0.00)	0.268	0.009	***	(0.00)	0.003
Post-secondary enrolment	0.066	***	(0.02)	0.001	0.051	***	(0.01)	0.00	0.025	**	(0.01)	0.065	0.029	***	(0.00)	0.000
Time	−.454		(3.48)	0.897	−.302	***	(0.09)	0.001	47.126	***	(4.89)	0.000	0.386	***	(0.07)	0.000
Time^2	0.256		(0.2)	0.199	0.059	***	(0.00)	0.000	−.454	***	(0.25)	0.000	−.029	***	(0.00)	0.000
Time^3	−.009	**	(0.00)	0.017	−.002	***	(0.00)	0.000	0.042	***	(0.00)	0.000	0.001	***	(0.00)	0.007
M1[Country]	1		(constrained)		0.565	***	(0.04)	0.000	0.397	***	(0.07)	0.000	1		(constrained)
(Intercept)	−4.676		(20.11)	0.22	−.617	***	(0.50)	0.000	−97.546	***	(30.48)	0.000	−.402		(0.36)	0.27
(Intercept 1 Model 4)													2.050	***	(0.07)	0.000
(Intercept 2 Model 4)													0.437	***	(0.08)	0.000
	Coef.		Std. Error										Coef-		Std.Error
var(M1[Country])	3.417		(0.661)										0.304		(0.07)
Number of obs	3144												3144
N of countries	131												131
Log likelihood	−988.06												−121.6142