International Organisations and the Proliferation of Scientised Global Reporting, 1947–2019

ABSTRACT

As novel instruments in global governance, international organisations’ (IOs) global reports have emerged in all policy sectors in the more recent period. Drawing on an original dataset of N = 363 editions from N = 95 explicitly global reports from the period 1947–2019, this study documents the rise of reporting and uses citation and content analyses to examine the changing role of science. Reporting based on scientific research and quantitative indicators increases over time and across all sectors, yet particularly striking since the late 1980s and most in sectors dealing with human development and the environment. Drawing on arguments from world society theory, the sociology of quantification and post-truth approaches, this work argues that while reports provide IOs with new legitimacy in science-based governance, their scientised and quantified nature is likely to make IO activities the target of antiscientific populist rhetoric and critical arguments about a reductionist interpretation of science.

KEYWORDS:

• Global reporting
• international organisations
• world society
• quantification

Introduction

The importance of scientific knowledge in national and global policymaking has grown considerably at the beginning of the twenty-first century. Political discourses now routinely centre on policy-relevant evidence and demand scientific work with impact (Drori et al. 2003; Watermeyer 2019). Indicative of such a knowledge turn in global discourses, international organisations (IOs) have transformed themselves into research powerhouses publishing in high-quality academic journals, often in collaboration with academia (Zapp 2017a). Against the backdrop of this shift towards a more research-oriented role of IOs, this paper focuses on the emergence of global reports with a strong scientific, that is, academic, character. These reports deal with a wide range of sectors and further strengthen IOs’ position as providers of policy-relevant knowledge.

The purpose of this study is, first, to trace the emergence and proliferation of global reports defined as IO-led attempts to describe and analyse numerous problems as part of a global governance structure. Importantly, the study aims, second, at tracking the scientisation of these reports, that is, their growing reliance on scientific attributes, in describing global problems.

The paper starts from the assumption that scientific evidence has become an important source of authority in national and global debates from educational reform to climate change and pandemic diseases—often promoted by international organisations (Baker 2014; Meyer et al. 1997). Far from being incorruptible and invulnerable to power politics games, it seems that science—especially in its quantitative or positivist form—nonetheless provides rationality and certainty in a global risk society whose fragility but also global interconnectedness have become more apparent since the late twentieth century (Beck, Giddens, and Lash 1994; Beck, Bonss, and Lau 2003). At the same time, the legitimacy of science has recently seen serious drawbacks by an emerging generation of populist leaders that either reject evidence or present alternative facts as the basis of decision-making (Mede and Schäfer 2020; Laclau 2005; Lewandowsky, Ecker, and Cook 2017; Schofer, Lerch, and Meyer 2018).

Against the backdrop of this complex relationship between science and politics and policymaking, this article examines the scientific transformation of IO-led global discourses across virtually all governance sectors. I argue that these discourses have gone from mostly normative and descriptive in an earlier post-World War II period to highly scientific and analytic in the more recent period. In order to gauge these changes, I focus on IOs’ global reporting activities. Global reports represent formalised accounts of world societal self-analysis and communication and display the substantive structure of global governance. They either focus on specific topics (e.g. ILO's Global Report on Equality at Work or WHO's Report on the Global Aids Epidemic) or groups (e.g. the Loombda Foundation's Global Widow Report or UNICEF's the State of the World's Children Report) or cover broad domains of world society (e.g. the UN Human Development Report or UNESCO's Global Educational Monitoring Report). In doing so, the status of these reports is unique as they constitute a common ground for discussion among a wide variety of stakeholders. While not an entirely new phenomenon in the international arena, global reports have seen strong proliferation in the past three decades.

Drawing on an original dataset of N = 363 editions from N = 95 explicitly global reports authored by both intergovernmental and nongovernmental organisations, this study documents the dramatic rise of global reporting in the period 1947–2019 across an impressive range of policy sectors including, among others, environment, poverty, human rights, security, health and labour policy.

Citation and content analyses are applied to examine the changing role of science in global reporting as indicated by the use of scientific references and quantitative indicators. Results show that, early on, global reports made almost no use of (particularly, scientific) sources and standardised indicators were applied very sparsely. This has fundamentally changed in the mid-1990s and early 2000s when citations and the use of indicators soar up. With very few exceptions, a typical global report now uses up to 400 citations, most of these academic, and more than 30 indicators. Some reports such as the UN Global Environment Outlook (N = 7786 citations) or the World Bank's World Development Report (N = 4452 citations) resemble veritable sectoral meta-reviews. In general, this scientisation trend is particularly strong in reports dealing with issues of environment, general human development and education.

The article discusses the rise of scientised reporting as reflecting the trend of science-based global governance that opposes a burgeoning political rhetoric questioning the role of science but also global integration as a whole

Reflecting IOs’ transformation into epistemic actors that produce, synthesise, disseminate and teach scientific knowledge in the context of a global audience, global reports need to be seen as formalised and quantified accounts of the natural and social world (Espeland and Stevens 2008). In this data-driven logic, there seems to be little place for uncertainty and political power, nor for democratic legitimacy. Yet an ethics of quantification is needed not only to assure the responsible use of data and methods but to restore trust in science which, while becoming more important in global governance and policymaking, has come under fire by populist leaders and is increasingly at risk from suffering lasting legitimacy loss in the post-truth era.

The legitimacy of science in world society and the populist backlash

Late twentieth century social theory has emphasised the role of knowledge, science and reflexivity as the most essential institutions in modern societies. The so-called world polity or world society theory with its large body of empirical work (Meyer et al. 1997; Frank and Meyer 2020) is particularly helpful in documenting the growing importance of science in modern societies.

The key tenet of world society theory is to analyse Western society as a “cultural project organizing human activity to forge the proper links between the moral and natural worlds” (Meyer, Boli, and Thomas 1987, 2). In this view, Society and the State have been organised as purposive entities or as means to the ends of progress and justice. This process of social organisation is referred to as rationalisation describing “the structuring of everyday life within standardized impersonal rules that constitute social organisation as a means to collective purpose” (ibid.: 29).

Science, with its built-in universalism and expansive scope, is key in such large-scale rationalisation. While recent political discourses worldwide stress innovation and excellence as drivers of the knowledge economy, world society theory holds that the centrality of science in modern society stems in good part from its cultural significance as opposed to the more functionalist understanding in prevailing economic theories and political rhetoric (Schofer and Meyer 2005; Baker 2014). Science constructs social reality by supplying the ontological and epistemological frames for human action, interpretation and identity (Frank and Meyer 2020). Science transforms and thus constructs society by immersing objects and subjects in a general and (often) abstract collective reality, it legitimates and certifies specialised knowledge and personnel and indeed entire social sectors within a rationalised cosmological frame.

Historically, after two World Wars, which delegitimated aggressive nation-statehood and which saw the rise of a culturally liberal United States as a dominant global power, the nascent world society was built on a universalistic cultural frame rooted in the natural laws of science and human rights. A distinctly global social order emerged, to which the university and science are central as they provide a quasi-religious basis for systems of legitimacy, meaning, as well as individual and collective action (Baker 2014).

Indeed, empirical analyses in the world society tradition convincingly show that the role of scientific knowledge has grown in importance as indicated by striking growth in postsecondary enrolment, number of universities, science associations, research policy and research output as well as evidence-based policy-making initiatives (see Zapp 2018 for a review). Scientists now produce authoritative evidence and advance opinions in both national and global debates, from the rights of dolphins to climate change and educational reform, often channelled through and in exchange with international organisations and social movements (Suárez and Bromley 2012).

The growing importance of scientific knowledge in modern societies notwithstanding, science is constantly threatened by political ideologies and, more recently, populist rhetoric (Handmer and James 2007; Schofer, Lerch, and Meyer 2018). Since the mid-2000s, a new class of populist leaders has entered the political arena around the world promoting an essentialist, often mystified, will of the people, an exclusionary stance toward elites, outsiders and minorities and a good deal of scepticism, if not, aversion, toward scientific knowledge (Mede and Schäfer 2020; Laclau 2005; Lewandowsky, Ecker, and Cook, 2017; Norris and Inglehart, 2019). These agendas clash with the method and language of science where sober analysis are central and international collaboration and mobility have turned science into a global and, indeed, globalising force (Zapp and Lerch 2020).

It is, therefore, not surprising that populist leaders routinely rebuke academia and examples of overtly anti-scientific populist leaders abound, not just since but even more so amidst the Covid-19 pandemic; wherever populist leaders are in power, they defy scientific expertise while fuelling conspiracy theories and distrust in science (Resnick, 2020; Weible et al. 2020).

At the same time, many populists use their own “evidence” to support their claims. In mild instances, populist leaders are highly selective in the science on which they draw to support their agenda. In more extreme cases, they employ fake science and pseudoscience to undermine people's trust in data, information, and facts (Hopf et al. 2019; Wang Huang 2021; Ylä-Anttila 2018). Climate change, vaccines, genetically modified food, evolutionary biology and 5G, among other topics, are not only battlefields in contemporary public discourses they also represent breeding grounds of “alternative facts” sometimes described as examples of a beginning “post-truth era” (Allcott and Gentzkow 2017; Barrera et al. 2020; McCright and Dunlap 2011; Durnová 2019; Jasanoff and Simmet 2017; Lewandowsky, Ecker, and Cook 2017).

With IOs representing hubs of rationalised planning in global governance and advocates of universalistic scientific advice over political ideology and national populism in addressing global problems, they also come under fire by the same political leaders. The following section describes this role of IOs as “rationalized others” and “epistemic actors” and the thorny relationship with nationalist political strongmen.

International organisations as rationalised others and epistemic actors

Traditionally viewed as tools of powerful states or functional responses to reduce transaction costs in complex international relations as in (neo)realist, critical and liberal international relations scholarship (see Zapp 2017b for a review), this work understands international organisations as autonomous and authoritative actors in world society. Their legitimacy and authority rest primarily on their status as “rationalised others” and “epistemic actors” who produce disinterested and policy-relevant scientific knowledge. I elaborate on these two features.

International organisations, both intergovernmental and nongovernmental, routinely defend the “good” and “noble” in global debates, most prominently freedom, justice, peace and progress. They advocate fighting poverty and diseases and the banning of landmines and illegal hunting, the call on states to opt for diplomacy over war and spearhead global action against climate change (see Boli and Thomas 1999; Lechner and Boli 2014 for a review). To emphasise this quality, world society theory refers to these international actors as agents of wider cultural goods or “rationalised others”—a reference to Mead's generalised others who serve as a source of expectation of how to act in society (Meyer et al. 1997, 165).

While IOs derive much of their legitimacy from the fact that they raise their voice against infringements of the global moral order, they add to this an epistemic or scientific dimension by hosting large numbers of experts and accumulating stocks of rationalised and policy-relevant knowledge. Indicative of their role as knowledge hubs, IOs are key actors in contemporary policymaking which has been marked by sweeping trends of quantification, measurements and rankings (Espeland and Stevens 2008; Klees and Edwards 2014; Meyer and Benavot 2013; Power 2004). As IOs are also in charge of collecting, centralising and storing world data, they inevitably contribute to such governance (some might critically say, tyranny) by numbers (Jang, Cho, and Drori 2014; Muller 2018; Supiot 2017).

Across sectors, IOs drive global agendas and advance goals which are to be measured cross-nationally (Chabbott 2003). From the Millennium Development Goals to the Sustainable Development Goals, IOs have increasingly assumed the role of knowledge brokers aided by a large number of associated or in-house research institutes (e.g. UN System Staff College), which have been established to support this new role (Hwang 2006; Steiner-Khamsi 2009).

Indicative of such concentrated expertise, IOs’ scientific output in academic journals has seen a striking increase in the past two decades and they often bridge academia, policymaking and practitioners through a growing number of conferences, initiatives and partnerships and the creation of policy templates and best practices (Zapp 2017a). It is important to note that this transformative and intermediary role of IOs between science, policy and practice adds a new quality to current governance. What would otherwise be mere scientific growth becomes a qualitative novelty in the significance of science in world society. IOs increasingly promote science as empowering individuals, organisations and nation-states.

As showcase “instruments of shared modernity” (Meyer et al. 1997, 164), IOs almost ideal-typically enact, carry and disseminate rationalised world cultural goals, which makes them preferred targets of nationalist, populist and anti-scientific political leaders who question global integration by attacking IO staff, withdrawing from international agreements, and even cancelling IO membership (see Copelovitch and Pevehouse 2019 for a review).

Perhaps as a strategy to counter attempts of political instrumentalization and to present impartial and fact-based advice against political ideology, many IOs now publish global reports, which target a diverse audience and display IOs’ qualities as others and epistemic actors. The following section describes the features and purposes of formal global reporting.

What is a global report?

As part of the general trend of rational organisation in modern societies, formal and standardised reporting has diffused rapidly in the twentieth century and constitutes a key element in proper organisational actorhood and accountability (Bromley and Meyer 2015). Formal reporting usually presents information in an organised format for a specific audience and communicates key priorities of the organisation, its employees’ tasks and operations while aiding leadership and stakeholders in decision-making.

By contrast, global reports do not refer to the authoring organisations’ activities (unlike IOs’ annual reports) but describe general human activities and collective concerns that emanate from these. For example, overfishing and illegal logging, child poverty and drug violence reflect social problems that exist in all modern societies (albeit to a varying degree) or even specifically transcend national borders. The fact that global reports address activities and phenomena external to those of authoring organisations largely prevents them from merely serving as facade devices or bureaucratic propaganda as stated in some classical contributions to the study of organisations (Altheide and Johnson 1980; Meyer and Rowan 1977). Issues of window-dressing and decoupling between formal and activity structure can never be completely ruled out in organisational behaviour, yet the costs and structural efforts as well as the othering involved in reporting suggest that reports may be used to increase visibility and science therein being employed to warrant legitimacy.

Employing scientific knowledge does, however, not make them immune against evoking dramatism and crisis language in framing issues within these reports (Broome, Clegg, and Rethel 2012). As this study is interested in the scientific character of these reports, I consider such rhetorical features more as stylistic shell than substantive matter.

Further, global reports do not have a specific audience (such as stock holders) but are often produced for an “informed audience” in general. Obviously, policymakers and administrators, experts and scientists, teachers and practitioners are among the principal targets of global reports, yet many of these reports deliberately address all kinds of readers, available in the main world languages and free to download. Many of the larger reports are presented at press conferences, contain executive and lay summaries, they come with online tools, mobile phone applications and even cartoons (Zapp 2017b, 2020).

As periodical publications, report themes change yet cover these themes within the same governance domain or policy sector (e.g. human rights). They usually contain a large amount of statistical data and visualisation and are written by a mix of in-house and external staff and some organisations have special task forces or even entire units dedicated to preparing reports. These units invite specialists with particular expertise in the field covered by the respective report.

Preparing a global report usually takes multiple months, in some cases even several years, meaning that even before one report is published the next one is already on the drawing board (Zapp 2021). Their funding is either covered by internal budgets or backed by consortia specifically created for the report involving a wide range of funding sources and stakeholders (e.g. governments, other international and for-profit organisations).

In many cases, such external funding is preferred as it warrants greater autonomy from the stance of the wider organisation, which often needs to be more diplomatic in presenting findings and formulating recommendations as it depends on country contributions (consider for example the case of Israeli and U.S. withdrawal from UNESCO due to its recognising Palestine and the recent decision by Donald Trump to end WHO membership).

Global reports bring IOs’ two main characteristics together; first, they address issues of general concern, only indirectly linked to the organisation's own operations, instead dealing with the human species and planet earth as a whole and, second, they do so in a systematic and analytical, that is, scientific fashion. As mentioned earlier, IOs have grown their scientific output through, for example, academic publications to a striking degree in the past three decades (Zapp 2017a). It seems plausible to assume that such scientific activity and knowledge is now also applied and implemented in their reporting activities.

The purpose of this study is (1) to identify the range of human concerns, i.e. governance sectors, to which global reports relate and (2) to gauge the extent to which the importance of science in these reports has increased over time. Reflecting the intensification of global goal-setting and related monitoring as well as IOs’ internal research orientation, I expect that both the array of policy sectors and the scientific nature of reporting have expanded considerably in the more recent period. In the following section, I describe the methodology used in this study.

Data and methodology

In order to identify global reports, this study relies on the Yearbook of International Organisations (YIO) published by the Union of International Associations (UIA). The YIO represents the most comprehensive and authoritative source for the study of all kinds of international organisations (IOs) (UIA 2019). The YIO database contains information on IOs’ main structural properties including their publications along with information on title and year of publication. Reports are only included if they represent a periodical or recurring publication, which excludes many single-year reports of global scale. This strategy was chosen, first, because the study is interested in documenting institutionalised and recognised domains of world society as opposed to transitory efforts with little resonance in the wider governance community and, second, in order to be able to trace changes in reporting practices over time.

Search for reports was based on report titles containing very general strings that indicate a universal coverage such as “world”, “global”, “earth”, “planet”, “international”, “human”, “countries”. Reports were then downloaded from the organisation's official website and examined as to whether they actually cover global phenomena (e. g. through table of contents, indicators, section content). This strategy may not warrant exhaustive retrieval as some reports may have changed their coverage (and title) over time, while others might simply not refer to a global coverage in their title, yet offer such coverage in their content. Indeed, the true number of reports with a global scope may well exceed the sample size presented in this work. It may also exceed a realistic sample size for a thorough qualitative analysis, particularly the coding work performed (see below). I chose greater validity and analytical rigour through systematic data collection over large-N. Assuming an otherwise larger number of potential reports with a global scale does, however, help underscore the importance of the phenomenon of global reporting in general and further research may widen the analytical scope in this regard.

Whenever more than one organisation authors a report, reports are counted once while authorship is assigned to both organisations. Organisational types are taken from the YIO definition and include intergovernmental, non-governmental, for-profit, and research organisation as well as a residual category aggregating such types as forums, trade unions and so on (see Appendix B for definitions). While there might be some analytical thrust in analysing these types as distinct entities (given their different mandates, funding, activities and rationales), I aggregate these based on their common features as internationally-oriented, engaged in reporting and involved in global regimes, organisational fields or governance sectors whose constitutive feature is to bring together a diverse set of actors (Chabbott 2003; Yeates 2014).

The initial sample contained N = 110 reports issued by N = 63 international organisations in the period 1947–2019. Of these 110 reports, 15 reports had to be excluded during further analyses as I could not retrieve editions for each 5-year interval which was used in analysing the longitudinal change in citation and indicator practices (see below). The final analytical sample comprises N = 95 global reports (see Appendix A for a full list of reports and Appendix B for descriptive statistics on the organisational sample). Analysis followed a two-step process. In a first step, reports were aggregated to sectoral categories based on thematic similarity (e.g. human rights & democracy & law). These sectoral categories partly stem from UIA's (UIA 2019) definitions and, partly, represent original coding families retrieved through the qualitative analysis of reports (Corbin and Strauss 2008) and informed by previous research on IOs’ governance sectors (Boli and Thomas 1999; Yeates 2014). The sample of reports is represented by N = 14 coding families or sectoral categories (see Appendix A).

In a second step, citation analyses were performed manually during a multi-month coding process supported by nvivo.¹ References were coded as scientific whenever the citation refers to markers of academic publishing. These include journal or book title, digital object number, international standard serial number, or academic locales (e.g. institutes, universities, data centres). Sources were coded as non-scientific whenever these technical elements were missing and other sources are listed. This latter group of non-research-based references has been further differentiated as self-references, IGOs, INGOs, for-profits as well as government bodies, newspaper/ news agencies and a residual category of “others” where affiliation or organisational type were not clear.

Indicators were also coded manually and counted whenever observed values of concepts expressed as goals, targets, thresholds and other measurable units (usually along finite scales) are presented in a report (e.g. GDP/ capita for economic wealth). Indicators measure conceptual constructs and, depending on the complexity of the construct, are sometimes aggregated in indices. In those cases, indices were disaggregated and indicators were counted individually.

Reports were coded in 5-year intervals, that is, only one edition from all reports per interval is included in the analysis. In fact, the number of reports and the frequency of report editions as well as the citation intensity increase to such a degree in the recent period that complete coding becomes virtually unfeasible.

Results: the rise of scientized reporting

The number of reports grows slowly until the early 1990s when numbers soar up and remain stable until today (Figure 1). The first global report appears in 1947 with FAO's The State of Food and Agriculture, then Amnesty International's Annual Human Rights Report (1961). The most recent reports in the sample include FAO's State of the World's Biodiversity for Food and Agriculture, the UN Global Report on Environmental Rule of Law and the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services’ Global Assessment Report on Biodiversity and Ecosystem Services (both from 2019).

Figure 1. Publication of global reports, by year of first edition, 1947–2019 (N = 95).

While the expansive trend is a general phenomenon, report frequencies vary across sectors. Table 1 shows the prevalence of reports by sectors and tracks their expansion during the observation period. While democracy, law and human rights as well as agriculture and food represent the earliest domains of global reporting, human development and empowerment account for the largest group over time (N = 25). The UN Human Development Report, UNICEF's The State of the World's Children and more recently, the Loomba Foundation's Global Widows Report are some examples of reports from this sector.

Table 1. The expansion of reporting sectors, by years of appearance, 1947–2019.

	1947–1965	1966–1975	1976–1985	1986–1990	1991–1995	1996–2000	2001–2005	2006–2010	2011–2015	2016–2019	Total
Human development & empowerment			1	2	1	3	3	4	10	1	25
Environment, biodiversity & animal rights					2	2	1	2	3	3	13
Democracy, law & human rights	1	1		1		2		2	3	1	11
Macro-economic, finance, trade			1		2	1	2	2	1	1	10
Health & well-being				1	1	3	1	2	1		10
Security & safety & crime prevention			1			1	1	3	1		7
Migration, mobility & tourism						1	1	1	3	1	7
Energy & resources				1			2	2	1		6
ICT							1	1	3		5
Agriculture & food	1			1		1			1		4
Education & science							2	2			4
Humanitarian & crisis					1			1		1	3
Work							1		2		3
Culture					1			1			2
Total	2	1	3	6	8	12	17	22	30	9	110

Reports covering issues of environment, biodiversity and animal rights (N = 13), human rights (N = 11), health and well-being as well as economic issues also represent strong sectors (N = 10 respectively) with momentum since the late 1990s. Sectors that appeared more recently in reports include education and science, work and information and communication technology (ICT).

Turning to citation practices in global reports, Figure 2 presents data on total and average citation counts in global reports. Referencing is not a wide-spread practice until the 1980s when both the total and average number of citations increase considerably. The momentum is even more pronounced after 1995 when reports, on average, cite more than N = 150 different sources and the total count exceeds 10,000 citations. The upward trend remains strong and citations more than tripled in the period after 2000 now reaching well over 33,000 total citations for reports in the observation interval 2015–2019.

Figure 2. The expansion of citations in global reports, 1947–2019.

Total and mean citation counts say little about who is referred to in global reporting. Citation increases per se do not corroborate the scientisation hypotheses as reports can refer to a wide range of sources not necessarily including academic or scientific references proper.

Table 2 provides citation counts by type of source in the period 1947–2019. As a general finding, citations from all kinds of sources increase, from a mere 21 before the 1950s to a total of 33,301 references in 2019. Authoring organisations become more self-referential as does the field of international organisations as a whole indicated by a rise in self-references and references to IGOs and INGOs. At the same time, authoring IOs also draw on national sources such as policy reports and information from national statistical offices. They also rely on for-profit and research organisations as well as media agencies in preparing their reports. By far the strongest increase can, however, be found for academic sources, which account for the largest bibliographic part now totalling 22,288 citations.

Table 2. Citation counts by type of source, 1947–2019.

	1950	1960	1970	1980	1985	1990	1995	2000	2005	2010	2015
Self-references	12	25	22	21	67	201	354	708	844	1527	1819
IGO	5	9	25	20	232	525	674	1271	2760	4020	5041
INGO	0	0	2	4	8	73	149	428	661	702	606
Government	2	7	6	5	151	216	137	404	984	1756	1707
Academic	2	12	170	147	829	856	1487	6091	10,297	18,129	22,446
For-profit	0	0	0	0	3	21	39	63	126	177	428
Other IO	0	0	2	1	8	21	9	41	117	243	356
News/ media	0	0	0	7	29	58	50	82	243	370	759
Mixed	0	0	0	0	13	28	21	206	82	152	139
Total	21	53	227	205	1340	1999	2920	9294	16,114	27,076	33,301

Figure 3 shows the relative weight of academic sources. While all other main sources decrease or stagnate, the share of academic research sees a first momentum in the 1960s and another, even more pronounced one, in the early 1990s.

Figure 3. Citations by main source type as share of total, 1947–2019.

Do reports from different sectors differ in their citation practices? The sectoral distribution is reflected in the citation intensity of individual reports from respective domains. Among the top 10 reports with the highest citation counts (including academic citations), N = 3 come from the environmental domain (Table 3). Others include general human development, education, economics and migration. Interestingly, natural and social domains are addressed with the same scientific epistemology.

Table 3. Reports with highest citation count.

IO and report	Total n of references
UN Global Environment Outlook (2019)	7786
IPBES Global Assessment Report on Biodiversity and Ecosystem Services (2019)	5580
WB World Development Report (2016)	4452
WRI World Resources Report (2019)	2903
UN Human Development Report (2019)	2689
UN World Cities Report (2016)	2349
FAO Status of the World's Soil Resources (2015)	2217
UNESCO Education for all Global Monitoring Report (2018)	1977
UNCTAD World Investment Report (2017)	1925
IOM World Migration Report (2017)	1851

Citation counts, even when disaggregated by sector or source, obscure the specific source on which reports rely. Table 4 below compiles the most cited sources in global reports, organised by the type of source. The UN system in general and the World Bank and FAO in particular are frequently used in reports. The OECD, though not itself a global organisation or producer of a global report, is also among the most important IGOs. Government bodies are often referred to, especially national statistical offices and the U.S. development agency, USAID. The International Union for the Conservation of Nature is among the top INGOs in this list and Harvard University with its prominent scholar Amartya Sen accounts for most citations as an individual academic institution and scholar. Other important sources include the consulting and accounting companies McKinsey and KPMG. The Economist and BBC News represent the most referred-to media outlets.

Table 4. Most cited sources in global reports, by type.

Sources	Type	Citations
OECD; UN; World Bank; FAO (including self-references)	IGOs	>1000
USAID; National Statistical Offices	government	>500
International Union for Conservation of Nature	INGOs	>100
Harvard University (particularly Amartya Sen); International Food Policy Research Institute	academic	>100
World Economic Forum	other	>100
McKinsey; KPMG	for-profit	>100
The Economist; BBC news	media/ news	>100

Turning to the use of indicators, Figure 4 shows that indicator practices largely follow the general publication pattern of reports and the evolution of citations. Only two reports do not make use of indicators in their first edition and these are from the period before 1960. All those reports published after 1960 immediately rely on indicators. It is a striking finding that increases in indicator reliance reflect the intensification of global goal-setting, most clearly after 2000 when the Millennium Development Goals became the overriding priority in the international community and even more so with the beginning of the Sustainable Development Agenda.

Figure 4. The use of indicators in global reporting, 1947–2019.

With very few exceptions, a typical global report now uses more than N = 30 indicators. Indicators are used across all sectors and phenomena including economic, social, political and health categories. They help in reporting on GDP, trade, income equality, but also quantify the number of terrorist bombings, livestock breed, life satisfaction and happiness. However, as with citations, indicator intensity varies across sectors and reports. Table 5 below identifies those reports and their sectors that make heavy use of indicators. As with the citation distribution, natural and human domains are equally measured scientifically. The UN Global Environment Outlook leads the frequency table, yet reports from education, health, tourism and gender follow close. General human development, mental wellbeing and trade are also topics in this list of the most indicator-intensive reports.

Table 5. Reports with highest indicator count.

IO and report	Total n of indicators
UN Global Environment Outlook (2019)	215
UNESCO Education for All Global Monitoring Report (2018)	203
WHO Report on the Global Aids Epidemic (2017)	174
WEF Global Travel and Tourism Report (2019)	113
WEF Global Gender Gap Report (2018)	101
IPBES Global Assessment Report on Biodiversity and Ecosystem Services (2019)	90
FAO Report on the State of the World's Plant Genetic Resources for Food and Agriculture (2019)	77
UN World Happiness Report (2018)	77
WEF Global Enabling Trade Report (2016)	71
UN Human Development Report (2019)	56

Discussion: global reporting as evidence-based global risk governance?

Parallel to and aided by the institutionalisation of national reporting systems (Jang, Cho, and Drori 2014), global reporting has rapidly intensified, propelled by the expansion of collectively-agreed upon goals at the global level (Hwang 2006). The surge in the number of reports is particularly strong in the early 1990s and reasons for this might include the advent of wide-spread information and communication technology, the end of the Cold War and the associated effort to strengthen multilateralism. Another reason might be found in the emergence of some important global initiatives in environmental protection (e.g. Rio Summit in 1992), education (the launch of the Education for All initiative in 1990), children's protection (1990 World Summit for Children), and human rights (World Conference on Human Rights in 1993). These aspects of global development have long been the mandate of IOs (Boli and Thomas 1999; Lechner and Boli 2014), yet the rigour and complexity with which they are now dealt with through global reporting can be considered unprecedented.

It seems that once international initiatives have defined their agenda through global conferences they quickly move on to the next step of institutionalisation consisting of producing measurable objectives, which then easily snowball from one phase to the next. For example, the eight Millennium Development Goals were measured by 21 targets. Fifteen years later, in 2015, the Sustainable Development Agenda contains 17 goals and 169 targets encompassing social, economic and environmental dimensions. In principle, each of these 169 targets is meant to be measured and monitored by quantitative indicators.

Indeed, most reports are genuine repositories of human knowledge. For example, the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services’ Global Assessment Report on Biodiversity and Ecosystem Services has an impressive 1,800 pages, its preparation involved 400 experts from 50 countries and it draws on 15,000 scientific studies and government reports as well as 90 indicators (IPBES 2019). Some older outlets give prescient examples of the expansion over time. The first WB World Development Report (WDR), published in 1978, had less than 125 pages and no scientific references. A typical WDR, as published more recently, might have up to 500 pages and an average of 850 scientific references. The WDR from 2019, dealing with the Changing Nature of Work, for example, had an astonishing 4452 references. The content and tone have also changed. While some early reports resemble summaries, essays and a series of statements, such normative and apologetic tone is now always buttressed with the armoury of science. These shifts are most notable in those reports that already emerged in the post-WWII period and provide interesting discourse-analytical opportunities for further research.

Obviously, science, as seen in the prolific use of indicators, has a very specific (some might say, reductionist) manifestation as a quantitative tool kit. This echoes insights from critical policy studies and the sociology of quantification where science and statistics are sometimes seen as technologies of governmentality or at least insufficient tools for the job (e.g. Galaz et al. 2016; Klees and Edwards 2014; Muller 2018; Steiner-Khamsi 2009). While having clear merit, these critiques would also need to acknowledge that it is countries themselves that demand measurable goals and policy advice and commission IOs to come up with a state-of-the-art review to facilitate decision-making instead of IOs governing top-down via indicators. Here, statist models of counting and accounting are rescaled from the local and national level where they originated and become transposed to the global arena promising management by objectives on an international scale (Desrosières 1991; Supiot 2017).

This is not to say that the governance by numbers does not require an ethics of numbers (Espeland and Stevens 2008). With an abundance of data, the quantification of the natural and social world and the measuring of planetary boundaries rapidly progress (Galaz et al. 2016). While the creation of indicators is an immediate consequence of such quantification, it certainly propels the proliferation of indicator-based ratings and rankings further down the line. These qualitative applications of quantitative measures can, in turn, entail grave consequences. For example, World Bank and International Monetary Fund lending is tied to indicator-based conditionality as is official development cooperation and philanthropic funding (Zapp 2017b). Another example is the growing availability of big data in social and health domains at a global scale and the recent Covid-19 pandemic has seen a number of critical responses from policy scholars on the use of large-scale quantitative reporting during the crisis (Saltelli and Di Fiore 2020).

The precise mechanisms of a possible quantification ethics are yet to be defined, yet some key principles have been suggested. Among these, reflexivity about the societal and political impact of quantification, the boundaries of knowing and not-knowing, and a discussion about the quality of data and algorithms may be promising perspectives on a new data/science—policy contract (ibid). At the global level, such thoughts have become part of a burgeoning discourse about the implications of large-scale data in global governance. For example, the UN Global Working Group on Big Data for Official Statistics has developed charters that attempt to responsibly frame the use of big data at the international level (e.g. the Bogota Declaration and Kigali Declaration) (UN Stats 2020). Since the mid-2010s, the UN also organises international conferences on the role of big data where ethical questions, while not the central theme, are also touched upon.

It remains to be seen how much of these debates is translated into a veritable code of conduct that extends into the use of quantitative data more generally and that reaches the multitude of IOs described in this study. In the meantime, absent a more viable (just?) basis of decision-making and despite the lack of collectively-shared guidelines, quantitative operationalisation will continue to thrive and variables will remain the main component of the reality constructed in global reports.

Such large-scale commensuration in global governance raises a number of questions, which need to be addressed in future research to embed this study in a wider context. Among these, it is important to recognise that defining features of science—its impartiality and objectivity—may well conceal reports’ selective and biased content as well as its normative undercurrents. Related to this, specific and sometimes idiosyncratic paradigms and world views associated with particular disciplines may dominate the reporting organisation and its outputs. Critical research on distinct rationalities, epistemic cultures and disciplinary myopia in specific research settings (including IOs) point to such biased knowledge production (Klees and Edwards 2014; Knorr-Cetina 1999). Finally, growing reliance on data-driven decision-making at the IO level is likely to deepen the lack of democratic legitimation, which has often been noted in international relations research (e.g. Tallberg and Zürn 2019). With technocratic and expertocratic modes of governance gaining more ground, it remains the task of further research to examine whether and how IOs manage to bridge the gap between science, policymaking and wider society.

Linking this sociological research on quantification and quantitative knowledge construction back to the debate on a beginning post-truth era, one may ask whether it is a mere coincidence that those reporting sectors that show the highest degree of scientific elaboration as measured through citation and indicators (e.g. the environment, human development and education) are also those that have seen the strongest political, and especially populist, resistance in recent years with a resurgence of nationalist rhetoric and foreign policy (Copelovitch and Pevehouse 2019). Most prominently, climate change denial and withdrawal from UNESCO by the U.S. and Israel seem to have prompted IOs to make even heavier use of science to prevent further populist backlash.

On a more general note, datafication might, thus, produce a puzzling paradox. While numbers are usually considered the basis of hard facts, reminiscent of hard science, their proliferation not only triggers calls for a more ethical discussion of their use but also their outright rejection or strategic manipulation by political, particularly populist, leaders (Hopf et al. 2019; Wang and Huang 2021; Ylä-Anttila 2018). It will become much more difficult to separate fake science, pseudoscience and “alternative facts” from honest evidence if, overall, all sectors of the social and natural world are becoming quantified (Allcott and Gentzkow 2017; Barrera et al. 2020; Dunlap and McCright 2010; Durnová 2019; Jasanoff and Simmet 2017; Lewandowsky, Ecker, and Cook 2017). As a consequence, for those not socialised in academic heuristics, the simplicity of political and populist rhetoric may still sound more convincing than the voice of reason.

The latter is certainly audible in the global reports analysed in this paper. Their conception of science echoes the main ideas of the Enlightenment as an empowering societal force that centres on the belief that individual and collective progress is not only desirable but achievable by the action of individual actors (Meyer, Boli, and Thomas 1987). Global reports provide roadmaps of development, codes of conduct for individual action and large-scale templates for how to organise society. Yet science can only be as empowering as individuals are science-literate and willing to prompt decision-makers to follow fact over opinion. The age-old debate about how to bridge the gap between scientific knowledge and policy-making is intensely held at IOs and considered even harder to resolve in a global than a national setting. Yet, this should not keep the involved actors from seeking answers as not only their own fate depends on how these answers will look like.

Conclusion and outlook

Drawing on a large sample of global reports, this work documents the growing role of scientific, that is, academic, knowledge in communicating, among others, environmental, social, economic and health problems of global scope. Aided by the standardisation and availability of data and the proliferation of collective goals, IOs have taken on, more than ever before, the role of knowledge hubs relying on the profound authority of science. These global reports amass, synthesise and communicate vast stocks of knowledge deemed crucial in addressing humanities’ ills. While these reports may reflect an unprecedented degree of reflexivity in world society, they might also convey a deceptive sense of systemic certainty and measurable progress in a rationalised world where disruption and non-linear development seem to be an unthinkable and unacceptable scenario and populist leaders offer alternative facts or reject evidence altogether. It is a striking paradox that reporting and its scientisation intensify at the same time as nationalist and populist rhetoric and action resurges. It seems that the more these nationalist and anti-scientific arguments enter the public debate, the stronger seem IOs to scale up their efforts to make themselves immune against politicisation through relying on impartial evidence but also to make themselves more indispensable through accumulating knowledge stocks which are unparalleled in world society.

Future research needs to address the shortcomings of epistemic or science-based governance, and specifically global reporting, in assessing how effectively IO knowledge trickles (back) into both academia and policymaking and how it helps in providing democratic legitimacy to IOs, overcoming political calculus as well as anti-science and nationalist rhetoric.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Mike Zapp

Mike Zapp is a post-doctoral researcher at the University of Luxembourg (UL), Department of Social Sciences. Before joining UL, he worked as a fellow at Johns Hopkins University and Stanford University and held a prior post-doc position at UL within the project The New Governance of Educational Research. His research is interested in institutional change in global (higher) education, international organisations and global governance and has appeared in such journals as International Sociology, Sociology of Education, the International Journal of Educational Development and Globalizations.

Notes

1 I need to thank Jurgen Muci for crucial assistance in coding.