126
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Performance-based funding and institutional practices of performance prediction

ORCID Icon
Received 19 Sep 2023, Accepted 30 May 2024, Published online: 26 Jun 2024

ABSTRACT

Since the financial crisis in 2007–2008, higher education systems across Europe have increasingly become subject to performance-based funding reforms. Performance-based funding in higher education thus constitutes one of the most tangible and widespread examples of how performance data are incorporated into governance instruments in educational governance, and in consequence, how a mundane management practice like budgeting has become a data practice. This article investigates how university managers predict their future performance in the context of budgeting. In order to understand the enacted relations between the present and the future implied in practices of prediction, the article analyzes the meticulous and often invisible practices of prolonging individual data points into the future in two case studies, encompassing long-term budgeting in a Danish and a Norwegian university. The analysis shows that these prolongation practices differ across the two cases, involving serial and continuous temporalities, respectively. However, in both cases, the prolongations are made up of various calculations, regulations, sensations and affectivities, which then become constitutive in the sense that the resulting long-term budgets severely restrict the room for maneuver of managers. The article thus conceptualizes these data practices as preemptively restrictive and prospective decision loops, affecting decision-making in the present.

Introduction

Since the financial crisis in 2007–2008, higher education systems across Europe have increasingly become subject to performance-based funding reforms as part of what has been labelled a common European ‘grammar’ for governance (Magalhaes et al., Citation2013). In performance-based funding systems, a share of the public funding of universities is determined on the basis of one or more performance indicators, such as the production of student credits (ECTS), the production of research publications and the obtainment of external funding, often combined into complex calculative formula determining how funding is allocated (Landri et al., Citation2017). Performance-based funding in higher education thus constitutes one of the most tangible and widespread examples of how performance data are incorporated into governance instruments in educational governance, and in consequence, how a mundane management practice like budgeting has become a data practice (Decuypere, Citation2021) rather than merely a financial exercise or solely a decision-making process. Working from the conceptualization of institutional budgeting within performance-based funding systems as a data practice, the question of how such data practices unfold constitutes an important yet overlooked aspect of the datafied governance of education. In contrast to most other data practices, which are concerned with data of the past or present, budgeting as a data practice is concerned with how performance data may develop in the future. Therefore, this article investigates how university managers predict the future performance of their departments in the context of budgeting.

The article is based on a 2-year-long research project, including fieldwork in a Danish and a Norwegian university, selected on the basis of their similar sizes, international rankings and characteristics as multi-faculty universities. In both universities, long-term or multi-year budgeting (or prognosis) is a mandatory exercise for university managers. Estimating or predicting the future performance of the university thus constitutes a mundane and recurring yet high-stakes task for university managers in the two case universities, as it does in universities in performance-based funding systems in general. As also recognized in the public administration literature (for example, Bahl & Schroeder, Citation1984; Forrester, Citation1991; Slawomir, Citation2012; Steunenberg, Citation2021), long-term budgeting is adopted in contemporary public university governance and management with the purpose of improving control and optimization. This purpose should be seen in light of the frugality and in some contexts even austerity characterizing educational governance in contemporary Europe (Nixon, Citation2017). Denmark and Norway are also characterized by such a frugality, however governed according to a stronger state-driven vision than other more marketized systems (de Boer & Maassen, Citation2020). Nevertheless, the policy-contexts of Denmark and Norway generate different intensities of frugality, thus rendering comparisons of case studies from the two countries relevant. The two case universities were each studied via a qualitative case study, focusing on the humanities faculty in each university (thus acknowledging the current economic pressure on the humanities in many countries), as well as a specific department within the selected faculty.

The article studies practices of prediction by examining predictive data practices from a topological perspective (Decuypere, Citation2021). Practices of prediction take place when performance data of the past and the present are in some way or another prolonged into the future. In order to understand the enacted relationships between the present and the future implied in practices of prediction, the article analyzes these practices in the two case universities. As the analysis will show, the prolongation practices differ across the two cases, involving serial and continuous temporalities, respectively. However, in both cases, the prolongations (as well as the underlying data) are made up of various calculations, regulations, sensations and affectivities, which then become constitutive in the sense that the resulting long-term budgets severely restrict the room for maneuver of managers. As such, the data practices in long-term budgeting can be conceptualized as preemptively restrictive and prospective decision loops, affecting decision-making in the present. The paper thus argues that prediction can be seen as a way of engaging the future in the present for present purposes.

With the focus on prediction practices in long-term budgeting in universities, the article adds to recent scholarship on time and temporality in the (datafied) governance of education (Decuypere & Vanden Broeck, Citation2020; Decuypere et al., Citation2022; Lingard, Citation2021; Lingard & Thompson, Citation2017; Madsen et al., Citationin press). The article contributes empirically and conceptually to this literature by exploring how predictions of, and by, performance data may take place, as well as methodologically by demonstrating how such prediction practices can be studied with social topology as an analytical and theoretical framework. Based on these contributions, the article offers a theorization of prediction as topological. Furthermore, the article contributes to scholarship on performance-based funding in higher education by highlighting an additional effect of the introduction of performance-based funding on the contemporary circumstances of higher education, namely the uncertainty implied due to its variation from year to year and the consequences of this on educational quality.

The article first outlines previous scholarship on performance-based funding and relates the main points to the two case countries and universities studied in the article. The article then introduces a relational theorization of numbers drawing on agential realism and (social) topology as the analytical and methodological frameworks of the paper. The theoretical framing also includes a methodological unpacking of the table, including its particular materialization as spreadsheets, as topological forms through which relations between budget numbers, including those describing the presence and the future, can be studied. The paper then analyzes two different practices of prediction characterizing the two cases: Perennial budgeting, which involves a year-by-year bottom-up technique of prediction characterized by a serial temporality, and long-term prognosis, which involves a trend-based forecasting technique characterized by a continuous temporality. Neither of these practices of prediction exists in pure forms in the empirical material, yet they appear as archetypical practices of prediction involving different calculative and affective practices that the analysis seeks to tease out. In the second part of the analysis, the article analyzes the affective and sensorial forces involved in both perennial budgeting and long-term prognosis. Finally, the paper offers a tentative topological theorization of prediction, which may inspire future studies of predictive datafication, as well as new insights for scholarship in performance-based funding in higher education.

Two cases of performance-based funding

Performance-based funding of universities has spread across Europe over the past three decades, in combination with other funding models such as tuition fees, earmarked funding, competition-based funding and the more traditional block grants (Pruvot et al., Citation2015). The diversification of funding, including an increasing reliance on formula-based performance funding models and performance contracts, has been successfully promoted by the European Union, the Bologna Process and the OECD (Landri et al., Citation2017; Magalhaes et al., Citation2013; OECD, Citation2010). While the funding models vary greatly across countries (Zacharewicz et al., Citation2019), research nevertheless displays a common trend of supplementing or replacing input indicators, such as student numbers, with output indicators, such as degrees, credits, assessments, publications and grants, as the basis for funding allocation (Magalhaes et al., Citation2013).

In line with the picture drawn in research, the Danish and Norwegian funding models also vary in terms of components and complexity. In Denmark, national funding for teaching and research is allocated according to two different funding models. The funding model for teaching includes a block grant (25%), an activity-based grant (65%) and a performance-based grant (10%) (Ministry of Higher Education and Science, Citation2017). The performance-based grant includes grants based on graduate employment rates and completion rates. While the activity-based grant is in principle input-based, as it comprises fixed rates (tiered according to subject areas of programs) multiplied with student numbers, the use of full-time equivalents (FTEs) in the calculation of student numbers, and thus the performance of students in terms of completing exams, in reality makes this part of the teaching funding performance-based, in addition to the explicitly performance-based component. The research funding model also includes a block grant (appr. 80%) and a performance-based grant (appr. 20%), based on share of teaching funding, share of external funding, research publications and doctoral graduates (the Danish Government, Citation2009). Since 2022, the research publication indicator is no longer measured but has been ‘frozen’ because of the administrative costs involved in measuring this indicator (the Danish Government, Citation2021). Nevertheless, in sum, Danish universities receive funding based on a number of performance indicators. This is also the case for Norway, where the national funding model besides a block grant includes a performance-based grant comprising eight indicators, including credits, degrees, doctoral degrees, exchange students, EU grants, national research council grants, other external grants and publications (Kunnskapsdepartementet, Citation2022). Whereas the teaching-related performance elements (the first four indicators) are granted on the basis of fixed rates, the research-related performance elements (the last four indicators) are granted as shares of fixed amounts, thus dependent on the relative performance of a university compared to all other Norwegian universities.

As also demonstrated in scholarship on research funding in other empirical contexts (Haake & Silander, Citation2021; Woelert & McKenzie, Citation2018), the Danish and Norwegian case universities have designed their own internal funding allocation models, slightly modifying the national funding models. The Danish case university allocates most of the funding to departments according to the Danish national funding model but has added an internal allocation model to distribute block grants, based on FTEs, external funding, doctoral graduates and publication. The Norwegian case university has adopted the same performance indicators as the Norwegian national funding model, but with different rates for most indicators, as well as a different weight of each indicator relative to the other indicators. Overall, student credits (or FTEs) and external research grants make up the most important performance indicators in both systems.

In addition to the national and institutional funding models, performance-based funding is also conditioned by other government regulations affecting the performance of universities. Such direct regulation of the higher education sector happens quite often in Denmark. Since the establishment of the current eight universities during the reform of the university law in 2003 and the following mergers, aimed at reorganizing the universities as self-owning publicly financed institutions with a sufficient volume and strategic capacity to function as independent organizations (Aagaard et al., Citation2016; Ørberg & Wright, Citation2019), the Danish government has reinforced the public control of the universities through a series of reforms. These include not only institutional accreditation but also a cap on graduate production based on graduate unemployment, a cap on English language courses, a set of instruments incentivizing increased student completion speed and a relocation of study places from large cities to the districts (Brøgger, Citation2021; Brøgger & Madsen, Citation2022; Brøgger et al., Citation2023; Madsen, Citation2022). These reforms are all forces constituting budgeting, as they affect the number and distribution of students in universities in various ways, as well as student activity. Norwegian universities have hitherto been subject to much less interference from the government, such as continuous regulatory reforms restricting the autonomy of universities (Pinheiro et al., Citation2019). Furthermore, Norwegian universities are legally and economically part of the state, yet semi-autonomous agencies free to dispose of their resources as they like and able to transfer money from one budget year to the next (Direktoratet for forvaltning og økonomistyring, Citation2023). This construction leaves Norwegian universities with slightly less economic autonomy, yet less tight economic constraints, than Danish universities. This difference in autonomy and control is also evident in budgeting practices in the two case universities. In the Danish case, the board of directors has formulated a bottom-line requirement for each department, entailing that the sum of all the items in the budget is required to achieve a particular numerical value, for example, zero or a surplus of 8 mill. DKK. In the Norwegian case, the sum requirement is more loosely formulated as an intention of achieving balance over a five-year period.

Literature on performance-based funding in European universities is relatively scarce. One body of previous scholarship has investigated performance-based funding model as a European policy adopted by most European countries (Landri et al., Citation2017; Magalhaes et al., Citation2013). For example, Landri et al. (Citation2017) labels the spread of performance-based funding a ‘standardization of differentiation’ across Europe. Another body of previous scholarship on performance-based funding has explored the effects of incorporating performance data in governance instruments on academic behaviors (Mathies et al., Citation2020; Rowlands & Wright, Citation2021), university strategies and missions (Zerquera & Ziskin, Citation2020) and status hierarchies among universities (Fadda et al., Citation2022). For example, research from Finland (Mathies et al., Citation2020) and Denmark (Rowlands & Wright, Citation2021) shows that the incorporation of performance indicators on publications in performance-based funding changed the publication behaviors of academics in the direction of more international high-ranked journals. Meanwhile, neither the implications of performance-based funding for managers in terms of uncertainty in budgeting nor the data practices necessitated to overcome this uncertainty, have yet been explored. Thus, this article picks up the question of performance-based funding from the point of view of managers and their predictive data practices.

A relational conceptualization of data

In order to examine the architecture (Decuypere, Citation2021) of prediction in the case of long-term budgeting, the paper conceptualizes budgets as made up of a number of individual data points, defined as ‘the concrete result of data practices’ and ‘the “sedimentation” or “snapshot” of what happens when activities or information (…) are captured, stored and represented’ (Decuypere, Citation2021, p. 68). Some data points are visible as the item-lines of the budgets, including, for example, ‘revenues’, ‘staff costs’, and ‘investments’, and added up to produce a bottom line. Others, including the predicted performance data, are built into calculation strings and thus invisible to the immediate eye yet co-productive of the visible data points. The combination of data points in a budget determines the ‘room for maneuver’ that is built into the budget, and thus imposes economic constraints on research and teaching. In essence, each data point therefore plays a role in determining what is possible and impossible in research and teaching.

Importantly, the data points of a budget are related to each other in a number of different ways, encompassing both what we could term nesting (where some data points are calculated by combining a number of other, underlying data points) and prolonging (where data points are stretched into the future). We are trained to perceive such relations between data points as made up of automated processes of pure, numerical and objective or disengaged calculation. However, as theorized as part of an ‘alternative ontology of numbers’ (de Freitas et al., Citation2016) and with inspiration from agential realism (Barad, Citation2007), ‘numbers, measurements, or quantitative data are a product of a myriad of forces’, and the question is ‘what might be some of the myriad forces that condition the production and enactment of these material-discursive phenomena’ (Dixon-Roman, Citation2016, p. 164). In the case of budgeting, data points estimating the future are thus not merely a linear product of calculations on data points describing the past and present. Instead, many other types of forces may play a role, including not only previous success rates of research applications and calculated dropout rates of students, but also social forces like affects flowing within and among the involved human beings. Whereas forces such as affects may themselves be theorized as products of a myriad of forces, this paper will first and foremost examine them analytically as forces affecting the generated data points, and thereby the predictions of performance conducted in long-term budgeting.

(Social) topology and prediction

In order to examine how data are prolonged into the future in long-term budgeting, the paper combines the agential realist theorization of numbers with a topological theorization of time. (Social) topology is a theory playing a limited but increasing role in educational governance research. It has first and foremost inspired studies of digitalization and transformations of national and global relationships (for example, Barbousas & Seddon, Citation2018; Decuypere & Lewis, Citation2023; Decuypere et al., Citation2022; Hartong, Citation2018; Lewis & Decuypere, Citation2023) but also studies of educational organization (Ratner, Citation2020b). Much of this scarce literature has focused on spatial relationships and demonstrated how contemporary forms of educational governance maintain or reinforce well-established topographical scales of nation states and international organizations as well as producing new spatial proximities of presence and intensity. A few studies have also adopted the theory of topology to study temporal relationships, for example, in relation to desired futures in educational discourse (Saari, Citation2022). Nevertheless, social topology has a great, yet untapped, potential to unlock new understandings of temporal matters, including predictions and contemporary engagements with the future, in data practices.

Like agential realism (Barad, Citation2007), the theory of topology entails that time and space are dynamic and composed relationships rather than pre-existing, static dimensions of the world. The theory of topology thus represents a break from scaled spatialities and linear temporalities (Decuypere et al., Citation2022). Instead of understanding proximity/distance as measurable in time and space, topology conceptualizes these relations through notions like presence and intensity for example, in terms of the tangibility of bygone pasts or potential futures. Based on this understanding, a topological–temporal relationship of prediction is enabled by processes of stretching, distorting, or folding data from the past in order to work up a particular future, thereby making the future present and forceful (Allen, Citation2016; Ratner, Citation2020b). As pointed out by pioneering scholars adopting the theory in education (Decuypere, Citation2021; Decuypere & Lewis, Citation2023; Decuypere et al., Citation2022), topology can serve as a methodological entry point for studying a given phenomenon. This can be done by starting from ‘relationality and how these relations (patterns, flows, articulations, orderings) have particular productive effects’ and by focusing on ‘the intensive (qualitative, topological) qualities of relations’ (Decuypere, Citation2021, p. 71). When it comes to prediction, the relations between data points estimating the future and data points describing the past and present are of particular interest. The qualities of future data points, including how they are generated as a product of calculations on past and present data points and the flows back and forth between present and future, are thus important relational aspects of numerical prediction.

The table as a topological form

An important entry point into how relations of proximity and distance in time and space are dynamic and composed is the topological form. A topological form is a plastic form that can be morphed into new shapes without any breaking, tearing, or suturing. While other studies highlight various topological forms, including lists, clouds, models, networks (Lury et al., Citation2012), the project (Lewis & Decuypere, Citation2023) and the graph chart (Madsen, Citationin press), I will in this article focus on the table as an important topological form in the quantification and datafication of education. Like other topological forms, the table is a very flexible form that can serve as a surface for relationships, represented by columns and rows. For example, if we turn to the popular case of performance testing of students conducted by transnational organizations, such as PISA, the table could mediate relations between countries (i.e. space) and different categories of scores in reading, mathematics, and science, or relations between a country’s reading performance in recent and previous test years (i.e. time). In the table as a form, the relationships between countries are plastic: Depending on the numerical value of the average performance score, countries may place themselves below or above the global average, as well as below or above other countries, thereby obtaining changing positions in the world. The continuous performative act of displaying changing relationships between, for example, countries makes the table an interesting topological form. As this paper will show, this is no less the case when we turn from transnational rankings to the mundane practices of prediction involved in budgeting.

The mediating role of tables is powered by spreadsheets and other digital software, which accommodate calculative relationships between cells and enable an immediate hierarchical sorting of data with just a few clicks. Spreadsheets furthermore turn tables hyper-topological, as they link tables to a multiplicity of other tables in ways where changes in a relationship (i.e. a numerical value or a calculation string) in one cell lead to various changes in relationships in other cells in the spreadsheet. In that way, spreadsheets are compositions of various types of data points and calculations, which the spreadsheets relate to each other in particular ways. In complex organizational contexts like universities, spreadsheets can be aggregated, and different aggregations allow one to alternate between different organizational or hierarchical levels. Furthermore, in the case of long-term budgeting, the spreadsheet includes a number of columns representing consecutive budget years. Hereby, the spreadsheet contains a seriality of different versions of the organizational unit that the budget concerns, all displayed through the same form and maintaining the same relationships between data points in the budget while simultaneously transforming the possibilities and constraints of education and research from budget year to budget year. Through these abilities, the spreadsheet co-produces particular spatial and temporal relationships of hierarchical organizations and consecutive years. Conjointly, these spatialities and temporalities form a grid of budget years for particular organizational units. Spreadsheets (and similar software) are thus important data infrastructures (Decuypere, Citation2021) that enable topological relationships to occur, both between separated and aggregated organizational units, between present and future and between management and the managed organization.

Based on this conceptualization, this article examines prediction through spreadsheets as a particular version of the topological form ‘table’. Meanwhile, bearing in mind the multiple forces involved in generating data points, spreadsheets themselves do not provide full access to the calculative practices involved in long-term budgeting. Therefore, the analysis of how data points are calculated, put in relation to each other, and prolonged builds not only on examinations of spreadsheets but also ethnographic fieldwork in the two case universities. In total, the empirical material encompassed interviews with heads of departments (or institutes) responsible for the budget, heads of sections, teachers and administrative staff members of the Faculty finance units involved in budgeting (27 interviews in total). In addition, the study included document studies of budgets and budgeting models as well as a single observation of a budgeting meeting.

The empirical material, all explicitly demonstrating the calculative practices involved in long-term budgeting in the two case universities, was generated through an explorative process, in which the empirical encounters were planned progressively throughout the research project until the empirical material concerning each emerging theme was saturated. The material was used to map data points and the (spatial and temporal) relationships between them, including how a prediction of a data point was achieved on the basis of historical data and other information. In addition, the interview material was used to gain insight in the implications of particular data practices, and quotes will be presented in the analysis to suggest these.

Prolongation practices: serial and continuous temporalities

A long-term budget includes a temporal dimension of consecutive years (columns) in addition to the categorization dimension of items (rows) that add up to the total sum. The underlying connections between data points across budget years indicate how data points are projected into the future. The prolongation of data points is a contingent process. The two case universities display two fundamentally different types of long-term budgeting: Perennial budgeting and long-term prognosis. These two types of long-term budgeting rely on two different kinds of calculative practices of prediction.

In this first part of the analysis, I will analyze the empirical cases in terms of the applied data prolongation practices. The two empirical cases are not pure, in the sense that they both draw on both types of calculative practices of prediction, yet each of them leaning on one of the prediction practices as a main practice and an ideal.

The Danish case: perennial budgeting

First, turning towards the Danish case university, the standard four-year budget is mainly calculated through perennial budgeting. Basically, this means that all four budget years are constructed in the same way: Each budget year is determined via calculations of data points (including future performance), which are estimated bottom-up, based on all known information and estimations concerning that year. In perennial budgeting, future data points are thus enacted by stretching the most updated available information and data in the presence into the future, thereby rendering the present intensely present in the enacted future (Decuypere, Citation2021).

For example, as a staff member from the faculty finance unit in the Danish case university showed me, the estimation of future performance on the indicator of full-time equivalents (FTEs) takes its starting point in last year’s ‘FTE production’. However, this historical number needs to be adjusted according to expected changes in student intake, perhaps because of changes in the program portfolio or the implementation of the national program-specific cap on student enrolment, as well as already known changes in the most recently enrolled cohorts. It also needs to be adjusted according to expected changes in drop-out rates and student activity rates (average ECTS per student per year), which are rates calculated for each program based on historical data, but sometimes adjusted due to known patterns or changes. The estimated average student activity rate is an underlying data point that is repeated in all of the coming budget years in the perennial budget. Nevertheless, this data point is only one among many types of information used to conduct bottom-up calculations of student FTEs in a given budget year.

Also external project funding is characterized by the prolongation of an underlying data point:

When we are working on the multi-year budget … we go look at some historical data, like how many [grants] do we usually get, in rough terms, and then we can go look at how many applications we sent out, and then we can cross our fingers and say that if there are ten percent of them that are successful, then it looks like we end up where we usually are – and then that is our best bet! … So it is in some way about daring to put your money on the event that the world probably is more or less like it usually is. (Interview with head of Faculty finance unit, Danish university)

The quote outlines the practice of predicting underlying data points via historical performance data in terms of a success rate. If a department has a history of obtaining 10% of the applied grants, then it appears safe to prolong this rate into the coming budget years. The rate is then multiplied with the budget sums of actually planned applications, thus bringing information of the present into the calculation.

With its bottom-up calculation based on prolonged underlying data points, perennial budgeting implies a serial temporality of consecutive, discontinuous budgets, each representing its own image of the space for maneuver. As a result, decisions on spending on operational costs and salaries are constantly made and remade, depending on what room for maneuver the current snapshot of the future provides. Unless the longer-term future shows a repetitive room for maneuver, only short-term investments and/or cuts can be made to accommodate the required balance of the budget, for example, by employing temporary staff, such as teaching assistants, rather than permanent staff, such as an associate professor, to cover teacher shortages. Thus, the future plays a restricting role on contemporary spending and thereby on the resources for providing quality teaching and research available to the department.

The Norwegian case: long-term prognosis

Moving on to the Norwegian case, this university in principle works with a one-year budget and an additional five-year long-term prognosis. In practice, at the time of the case study, many long-term prognoses were constructed much like perennial budgeting (i.e. calculated bottom-up for each year). However, members of the central administration of the university told me that they were implementing new principles for long-term budgeting, aiming to transform the calculative practices into long-term prognoses. As those principles were used by the Faculty finance unit in the case faculty, but not necessarily by the heads of departments, the prolongation practices were constantly subject to negotiation during my interviews. In order to tease out the differences between prolongation practices, the following description of long-term prognoses indicates the idealized practices of constructing a prognosis.

The basic idea of long-term prognosis is to predict through forecasting or historical trends, rather than through separate bottom-up calculations. For example, instead of calculating external funding by multiplying applied and planned applications with a historical success rate, the prognosis includes a standard or ‘dummy’ indicating the usual number of externally funded projects. The head of the finance unit explains the process:

Then we look at, okay, this is what we included as dummies or estimates, and then we discuss with the department if they think this is great, based on their history of national research council projects, EU projects, and other projects conjointly. (Interview with head of Faculty finance unit, Norwegian University)

As the head of the finance unit explains, the draft prognosis always includes a standard dummy of 1.5 granted projects per year for each department, representing the standard performance of departments as it looks historically. This dummy is based on historical trends alone, rather than present information on application patterns, and it is assumed relatively constant over time. The dummy can then be adjusted up or down if the historical data warrant this, however remaining a trend calculation of an estimated number of projects per year, rather a bottom-up calculation. Thus, in long-term prognosis, the future is enacted by stretching the past of historical data (rather than present information and data) into the future (Decuypere, Citation2021). In other words, the forecasting technique is not based on a current plan, like in perennial budgeting, but rather on previous reality displayed in historical data. The calculative technique of forecasting by prolonging trends in historical data and then adjusting for known or planned changes on top of the forecasts implies a temporality of continuity, rather than seriality.

Affective forces: the role of hope and fear in prediction practices

The prolongation of performance via a standard dummy described above represents the ideal of the prognosis approach. Nevertheless, as already indicated, the departments in many cases prefer the perennial budgeting approach of bottom-up calculations over the prognosis method. According to the head of the finance unit, this leads to unrealistic prognoses:

We are usually very optimistic when it comes to our spending … and in terms of income, we usually believe that we will obtain less revenues than what we get, so … This is something that we are currently working with in order to get what I call realistic prognoses. Of course, a prognosis is not a conclusion with double underlining – it has to change if things are changing, of course it has to! But as it is now, … people are scared of including extra income. So that is why I believe that the prognoses we are making now are not good enough to base the decisions they are making on, because they result in wrong decisions, strictly speaking, and most often in the direction that we do not dare to spend, and that we therefore are left with unused funding. (Interview with head of Faculty finance unit, Norwegian university).

As the head of the finance unit here explains, the heads of departments within the faculty (who she refers to as ‘we’) are driven by optimism, fear and lack of courage. In other words, prediction is not merely a product of calculations of numbers but also of affective and sensorial forces (Dixon-Roman, Citation2016). These affective and sensorial forces are the focus of this second part of the analysis.

Prediction as bodily sensation

The Danish head of the finance unit was one of the people talking about how estimation often requires sensation in my interviews:

When it comes to [long-term external funding], then we actually do not calculate. Then it is rather about the gut feeling of the head of department regarding what one believes and dares to believe. (Interview with head of Faculty finance unit, Danish university)

Gut feelings play an important role in prediction by hunch. But sensations are also important in relation to data points that are calculated on the basis of historical data. The data that are made subject to calculation are not merely given but can be composed in a number of ways, depending on what historical period that appears the most appropriate for prediction. Sensations are particularly important in cases of unstable historical data for example FTE rates that are declining, as a head of department explained:

Then you basically only have two things you can lean on. One is the historical development, and the other is your own ‘fingerspitzgefühl’ concerning if the trends you see right now are temporary or if they are permanent or structural. (Head of department, Norwegian university)

The ‘fingerspitzgefühl’ (or sensation in the fingertips) is crucial in the estimation of how to interpret historical data, even when such data are available.

The risk of estimating performance

As indicated by the quote from the head of the Norwegian faculty finance unit, the interviews furthermore show that fingerspitzgefühl’s and gut feelings are in most cases optimistic when it comes to the implementation of cost-drivers such as new initiatives and recruitment, but pessimistic and accompanied by affectivities of fear when it comes to achieving performance-based revenues dependent on student performance and research grants. In the Norwegian case university, the risk involved in expecting too many EU-funded research grants, despite a history of such grants over several years, is substantial because of a local incentive bonus accompanying research grants:

You get a quite substantial incentive bonus if you get an EU project … You will receive almost 80% of the expenses in your project budget on top as an extra bonus. So many do not wish to include an estimate on this besides those that are already known. Because if they include it as an estimate, then it will increase their estimated revenue or long-term revenue quite a lot. (Interview with head of Faculty finance unit, Norwegian university)

Being optimistic with such revenues is risky, because the sum requirement entails that the receiving department then plans for extra costs to spend the extra money, and the inclusion of such costs based on uncertain revenues generates fear of too optimistic budgeting.

In general, fear is the most dominant affect in budgeting – a point also made in the previous literature (Guénin-Paracini et al., Citation2014). In the case universities, the pessimistic gut feelings and affectivities of fear refer to four kinds of risks. First, the heads of departments are concerned with the risk of losing face by spending more than what they earn. As described by a staff member, they wish to ‘ensure that there is money left when they terminate their leadership term’ and thereby maintain their reputation. Second, they are concerned with the risk of overspending and thereby having to earn back the lacking funds through internal cuts, such as cuts on operational costs, postponement of refilling positions or (worst case) layoffs. In perennial budgeting, such pessimism and fears may be built into many data points, entailing that the budget becomes a stacking of risk margins, conjointly representing a far more pessimistic long-term budget than what the university economists consider a realistic prediction. Third, the heads of departments fear the reactions of their colleagues, who will have to cover their costs in case of overspending. And fourth, the heads of departments are concerned with the risk of losing funds if there is too much room for maneuver in the budget, as the surplus money may be redirected to other purposes. In perennial budgeting, this risk, based on a fear of cutbacks, is handled by adding unlikely, if not fictive, costs to the budget as a margin that serves as a buffer in case of low performance on other parameters. Such costs can, for example, include temporary positions like PhD and postdoc positions that can easily be cancelled.

The hope implied in strategic prediction

In addition to affectivities of fear, the prediction practices of perennial budgeting also include hopeful strategizing for example in the case of external project funding, as this head of finance unit explains:

External funding: can we continue getting as many successful research applications as we have had? And the competition just gets harder and harder … The majority base their budgets on an assumption of a slight increase, that is, a hope that … and that is probably both a kind of signal, that you want to communicate that well, we are getting stronger and better, so we will … In some way, it would be a shame to have a head of department that believes that the department was in decline. I mean, you have to believe in it! You have to set a goal, but on the other hand, you don’t want to set a completely unrealistic one. So most are budgeting with a sober increase. (Interview with head of Faculty finance unit, Danish university)

This quote displays that a ‘slight’ or ‘sober’ increase, driven by hopeful expectations of progress, may also constitute a way of estimating in perennial budgeting. This prolongation practice is based on a trend, which is more affective and strategic than calculated.

Affect in scenario tools

Turning to the technique of forecasting used in prognosis, the embedding of affectivities and bodily sensations into individual data points is challenged. Here, optimism and pessimism will be ‘revealed’ by historical trends. For example, an estimate based on historical grant levels deflate the affectivities of fear embedded in planning by taking into account that departments usually obtain more external funding grants than estimated, as well as the hopes that all spending plans are carried out. The forecasting technique, based on previous accounts, enables the head of the Faculty finance unit to question the cautiousness of budget responsible leaders:

But when we look at historical figures, then this is not the case! Our level of expenses is much lower there. (Interview with head of Faculty finance unit, Norwegian university)

By pointing out that spending will, statistically speaking, be less than planned for, while external funding will be more than estimated, the head of the finance unit can interrupt the hopes and fears embedded in predictions based on bottom-up plans.

Meanwhile, forecasting does also involve affect as a constitutive force, but in a different way. According to my interview with members of the central university staff, the Norwegian case university is working towards constructing various simulation and/or scenario tools as part of the prognosis process, wherein potential decisions and other changes can be tested out for their effect on the bottom-line. With such a tool, it becomes possible to ‘play with the numbers’ and use them more ‘smoothly’ to simulate various possible futures, thereby producing several scenarios to choose from:

This is where we are considering using simulations more, I mean, simply creating some scenarios for how it will go in a pessimistic, an optimistic, and a best estimate version. (Interview with support staff, Norwegian university)

The desired practices of simulating various economic scenarios thus also involves affectivities and bodily sensations but dislocates these in time to when the selection between various scenarios takes place.

Conclusion: theorizing prediction as topological

The exploration of how university managers predict their future performance in the context of long-term budgeting contributes conceptually and methodologically to scholarship on predictive data practices (Gulson et al., Citation2022; Lewis, Citation2018; Lunde, Citation2022; Ratner, Citation2020a). As demonstrated in the analysis, both perennial budgeting and long-term prognoses construct the future as highly continuous in relation to the present and past. However, they do so through two different techniques, including bottom-up calculations first and foremost based on the prolongation of all available information in the present and forecasting calculations first and foremost based on the prolongation of trends from the past. The resulting temporalities of seriality and continuity are both topological (Allen, Citation2016; Decuypere et al., Citation2022; Lury et al., Citation2012; Ratner, Citation2020b) in the sense that they establish the future as powerful and present. In both cases, the presence and intensity of the future is mediated by the table as a topological form (Lewis & Decuypere, Citation2023; Lury et al., Citation2012; Madsen, Citationin press) that has the ability to display the future visually right next to the present, thereby enacting it as qualitatively proximate. These empirically founded conceptualizations of temporal data practices as serial or continuous, as well as the topological form of the table, highlight that prediction practices are contingent and material practices that require situated investigation.

The relational conceptualization of prediction has furthermore enabled a theorization of prediction as a topological practice of enacting the here and now by engaging the future (via the past or present), rather than a practice of optimization – even though prediction is often narrated as such a practice (for example, Bahl & Schroeder, Citation1984; Steunenberg, Citation2021). The table as a topological form simultaneously enables a grid-like or static, chronological relationship and a plastic and dynamic relationship between the future and the present. We may term the second, plastic and dynamic relationship a decision loop, as it involves a lot of decisions (technical, sensorial and affective) that go into the construction of a future, which in turn constrains and enables decisions in the present. As the article has demonstrated, the specificities of data practices thus matter for how decisions are made in the present. Both the chronological and the loop relationship are temporal effects or a posteriori of the budgeting practices rather than a priori (Decuypere, Citation2021). While the double topological temporalities of chronology and the loop are particular for long-term budgeting, the relational conceptualization of numbers (drawing on de Freitas et al., Citation2016; Decuypere, Citation2021; Dixon-Roman, Citation2016) and the topological conceptualization of prediction (drawing on Allen, Citation2016; Decuypere et al., Citation2022; Lury et al., Citation2012) are relevant for studies of other kinds of datafication as well. It is not difficult to imagine how tables displaying educational performance based on large-scale testing mediate topological prediction practices of continuous temporalities similar to the trend-based prognoses analyzed in this article.

The article also contributes to scholarship on performance-based funding (Landri et al., Citation2017; Magalhaes et al., Citation2013) by zooming in on institutional practices and effects of performance-based funding. The two cases furthermore demonstrate that performance-based funding has implications not only for the behavior and well-being of academics (Mathies et al., Citation2020; Rowlands & Wright, Citation2021) and for the status of universities (Fadda et al., Citation2022), but also for the kinds of decisions that are made as a consequence of the uncertainty implied in performance-based funding, for the (mis)benefit of educational quality. The analysis has shown that the prediction of future performance is not merely a technical practice that enables managers to govern optimally (Bahl & Schroeder, Citation1984; Forrester, Citation1991; Slawomir, Citation2012; Steunenberg, Citation2021), but rather a process of actively (though oftentimes not deliberately) creating particular management circumstances for oneself via the used calculative techniques. In cases of a stacking of risk margins in the long-term budgets, building on fears and pessimism, we may speak of prediction as a practice of preemptive restriction, indicating an action of binding and thus obstructing one’s sight. By constructing data points that are laden with caution in the form of risk margins, heads of departments restrict their sight into the future and thereby they oftentimes obstruct their own room for maneuver. The case material, however, also includes examples of heads of departments that are optimistic. In these cases, we may speak of prediction as a practice of prospection, indicating an action of establishing a sight forward into a successful future. By building optimism into data points, heads of departments make room for decisions that were otherwise not possible. Forecasting practices work more explicitly with optimism and pessimism as matters of probability than perennial budgeting, however still resulting in preemptively restrictive or prospective practices.

Returning to the two case universities, it appears that while Danish universities are affected by much more restrictive policy circumstances than Norwegian ones (Aagaard et al., Citation2016; Brøgger et al., Citation2023; Ørberg & Wright, Citation2019; Pinheiro et al., Citation2019), heads of departments at the Norwegian university are no less preemptively restrictive in their long-term budgeting practices. The resulting consequences for education and research involve hesitant management and a preference for short-term investments that do not exceed the preemptively restricted future room for maneuver. These short-term investments include employing temporary teachers with less research experience; postponing recruitment for positions that fill gaps in required manpower and academic sub-disciplines; and conducting one-time investments in buildings rather than investing directly in teaching and research. Whether replacing perennial budgeting practices in which affectivity plays an important but non-explicit role with long-term prognosis and forecasting techniques will prevent the stacking of risk margins and create more sufficient rooms for maneuver in the future, only time will tell.

Acknowledgments

The author would like to thank participants of the network “Governing educational pasts, presents, and futures with data”, funded by the Joint Committee for Nordic research councils in the Humanities and Social Sciences (NOS-HS) in 2022–2023, and especially keynote speaker Mathias Decuypere, for contributing to the article’s conceptual and methodological framing of tables during the workshop series.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Independent Research Fund Denmark under Grant 0162-00038B; and the Joint Committee for Nordic research councils in the Humanities and Social Sciences (NOS-HS) under Grant 122266.

Notes on contributors

Miriam Madsen

Miriam Madsen is a scholar in the public administration of education, working with datafied and quantified educational governance. Her scholarship attends to the influence of economic methods and concepts in the quantification of higher education, and to the spatial and temporal dimensions of quantification and datafication, in particular in terms of how quantification practices differ across higher education systems embedded in different state models, and in terms of how the future is made known through quantitative techniques. She is leading the Nordic exploratory research network ‘Governing educational pasts, presents, and futures with data’ and lead editing the forthcoming special issue ‘Time and temporality in the datafied governance of education’ in Critical Studies in Education.

References

  • Aagaard, K., Hansen, H. F., & Rasmussen, J. G. (2016). Mergers in Higher Education : The Experience from Northern Europe, In R. Pinheiro, L. Geschwind, & T. Aarrevaara (Eds.), (pp. 73–88). Springer International Publishing.
  • Allen, J. (2016). Topologies of power: Beyond territory and networks. Routledge.
  • Bahl, R., & Schroeder, L. (1984). The role of multi-year forecasting in the annual budgeting process for local governments. Public Budgeting & Finance, 4(1), 3–13. https://doi.org/10.1111/1540-5850.00627
  • Barad, K. (2007). Meeting the universe halfway : Quantum physics and the entanglement of matter and meaning. Duke University Press.
  • Barbousas, J., & Seddon, T. (2018). Visualising the European space of education: Analytic borderland, transnational topologies and spaces of orientation. European Educational Research Journal EERJ, 17(6), 766–783. https://doi.org/10.1177/1474904118797732
  • Brøgger, K. (2021). A specter is haunting European higher education – the specter of nationalism. In V. Bozalek, M. Zembylas, S. Motale, & D. Hölscher (Eds.), Higher Education Hauntologies: Speak with ghosts for a justice-to-come (pp. 63–75). Routledge.
  • Brøgger, K., Degn, L., & Bengtsen, S. S. (2023). Danish university governance and reforms since the millennium: The self-governing university between state and institutions, the national and the global. Scandinavian Journal of Public Administration, 27(1), 9–28. https://doi.org/10.58235/sjpa.v27i1.11335
  • Brøgger, K., & Madsen, M. (2022). An affirmative-diffractive re-reading of the policy instrumentation approach through agential realism and the accreditation instrument. Journal of Education Policy, 37(6), 925–943. https://doi.org/10.1080/02680939.2021.1938239
  • the Danish Government. (2009). AFTALE mellem regeringen (Venstre og Det Konservative Folkeparti), Socialdemokraterne, Dansk Folkeparti og Det Radikale Venstre om ny model for fordeling af basismidler til universiteterne. https://ufm.dk/uddannelse/institutioner-og-drift/okonomi/tilskud/forskning-og-ovrige-formal/forskningsmidler-uni/copy_of_basismidler-efter-kvalitet/basismidler-efter-kvalitet/
  • the Danish Government. (2021). Aftale mellem regeringen (Socialdemokratiet), Dansk Folkeparti, Socialistisk Folkeparti, Radikale Venstre, Enhedslisten, Det Konservative Folkeparti, Nye Borgerlige, Frie Grønne, Liberal Alliance, Alternativet og Kristendemokraterne om: Basismidler til forskning. https://ufm.dk/uddannelse/institutioner-og-drift/okonomi/tilskud/forskning-og-ovrige-formal/forskningsmidler-uni/copy_of_basismidler-efter-kvalitet/basismidler-efter-kvalitet
  • de Boer, H., & Maassen, P. (2020). University governance and leadership in continental Northwestern Europe. Studies in Higher Education (Dorchester-On-Thames), 45(10), 2045–2053. https://doi.org/10.1080/03075079.2020.1823640
  • Decuypere, M. (2021). The topologies of data practices: A methodological introduction. Journal of New Approaches in Educational Research, 10(1), 67–84. https://doi.org/10.7821/naer.2021.1.650
  • Decuypere, M., Hartong, S., & van de Oudeweetering, K. (2022). Introduction―space-and time-making in education: Towards a topological lens. European Educational Research Journal EERJ, 21(6), 871–882. https://doi.org/10.1177/14749041221076306
  • Decuypere, M., & Lewis, S. (2023). Topological genealogy: A methodology to research transnational digital governance in/through/as change. Journal of Education Policy, 38(1), 23–45. https://doi.org/10.1080/02680939.2021.1995629
  • Decuypere, M., & Vanden Broeck, P. (2020). Time and educational (re-)forms-inquiring the temporal dimension of education. Educational Philosophy and Theory, 52(6), 602–612. https://doi.org/10.1080/00131857.2020.1716449
  • de Freitas, E., Dixon-Román, E., & Lather, P. (2016). Alternative ontologies of number: Rethinking the quantitative in computational culture. Cultural Studies ↔ Critical Methodologies, 16(5), 431–434. https://doi.org/10.1177/1532708616655759
  • Direktoratet for forvaltning og økonomistyring. (2023). Nettobudsjetterte virksomheter. Statsregnskapet. https://statsregnskapet.dfo.no/historier/2023/nettobudsjetterte-virksomheter
  • Dixon-Roman, E. (2016). Diffractive possibilities: Cultural studies and quantification. Transforming Anthropology, 24(2), 157–167. https://doi.org/10.1111/traa.12074
  • Fadda, N., Marinò, L., Pischedda, G., & Ezza, A. (2022). The effect of performance-oriented funding in higher education: Evidence from the staff recruitment budget in Italian higher education. Higher Education, 83(5), 1003–1019. https://doi.org/10.1007/s10734-021-00725-4
  • Forrester, J. P. (1991). Multi-year forecasting and municipal budgeting. Public Budgeting & Finance, 11(2), 47–61. https://doi.org/10.1111/1540-5850.00901
  • Guénin-Paracini, H., Malsch, B., & Paillé, A. M. (2014). Fear and risk in the audit process. Accounting, Organizations & Society, 39(4), 264–288. https://doi.org/10.1016/j.aos.2014.02.001
  • Gulson, K. N., Sellar, S., & Webb, P. T. (2022). Algorithms of education : How datafication and artificial intelligence shape policy. University of Minnesota Press.
  • Haake, U., & Silander, C. (2021). Excellence seekers, pragmatists, or sceptics: Ways of applying performance‐based research funding systems at new universities and university colleges in Sweden. European Journal of Education, 56(2), 307–324. https://doi.org/10.1111/ejed.12450
  • Hartong, S. (2018). Towards a topological re-assemblage of education policy? Observing the implementation of performance data infrastructures and ‘centers of calculation’ in Germany. Globalisation, Societies & Education, 16(1), 134–150. https://doi.org/10.1080/14767724.2017.1390665
  • Kunnskapsdepartementet. (2022). Finansiering av universiteter og høyskoler. Rapport til Kunnskapsdepartementet 17. mars 2022 fra et utvalg nedsatt 9. September 2021. https://www.regjeringen.no/no/dokumenter/finansiering-av-universiteter-og-hoyskoler/id2904847/?ch=7#id0070
  • Landri, P., Liumino, R., & Serpieri, R. (2017). Complex games of standardization in higher education: Calculemus. Bloomsbury Academic.
  • Lewis, S. (2018). PISA ‘yet to come’: Governing schooling through time, difference and potential. British Journal of Sociology of Education, 39(5), 683–697. https://doi.org/10.1080/01425692.2017.1406338
  • Lewis, S., & Decuypere, M. (2023). ‘Out of time’: Constructing teacher professionality as a perpetual project on the eTwinning digital platform. Tertium Comparationis (Print), 29(1), 22–131. https://doi.org/10.31244/tc.2023.01.02
  • Lingard, B. (2021). Multiple temporalities in critical policy sociology in education. Critical Studies in Education, 62(3), 338–353. https://doi.org/10.1080/17508487.2021.1895856
  • Lingard, B., & Thompson, G. (2017). Doing time in the sociology of education. British Journal of Sociology of Education, 38(1), 1–12. https://doi.org/10.1080/01425692.2016.1260854
  • Lunde, I. M. (2022). Learning analytics as modes of anticipation: Enacting time in actor-networks. Scandinavian Journal of Educational Research, Ahead-Of-Print, 1–15. https://doi.org/10.1080/00313831.2022.2123851
  • Lury, C., Parisi, L., & Terranova, T. (2012). Introduction: The becoming topological of culture. Theory, Culture & Society, 29(4–5), 3–35. https://doi.org/10.1177/0263276412454552
  • Madsen, M. (2022). Governing by numbers and human capital in education policy beyond neoliberalism: social democratic governance practices in public higher education (Vol. 19). Springer.
  • Madsen, M. (in press). (Un)certain and (ir)regular futures: Graph chart visualizations of forecasting in higher education. Discourse: Studies in the Cultural Politics of Education, pre-print ahead of publication.
  • Madsen, M., Lunde, I. M., Piattoeva, N., & Karseth, B. (in press). Introduction: Datafied temporalities and temporal modalities of data practices: Emerging foci in educational governance research. Critical Studies in Education.
  • Magalhaes, A. M., Veiga, A., Ribeiro, F. M., Sousa, S., & Santiago, R. (2013). Creating a common grammar for European higher education governance. Higher Education, 65(1), 95–112. https://doi.org/10.1007/s10734-012-9583-7
  • Mathies, C., Kivistö, J., & Birnbaum, M. (2020). Following the money? Performance-based funding and the changing publication patterns of Finnish academics. Higher Education, 79(1), 21–37. https://doi.org/10.1007/s10734-019-00394-4
  • Ministry of Higher Education and Science. (2017). New funding system in higher education [nyt bevillingssystem for de videregående uddannelser]. https://ufm.dk/lovstof/politiske-aftaler/endelig-aftale-nyt-bevillingssystem-for-de-videregaende-uddannelser.pdf
  • Nixon, J. (2017). Introduction: Thinking within, against and beyond austerity. Bloomsbury Academic.
  • OECD. (2010). Performance-based funding for public research in tertiary education institutions: Workshop Proceedings (1. Aufl. ed.). OECD Paris.
  • Ørberg, J. W., & Wright, S. (2019). Steering change - negotiations of autonomy and accountability in the self-owning university. In S. Wright (Ed.), Enacting the university : Danish university reform in an ethnographic perspective (Vol. 53, pp. 127–153). Springer.
  • Pinheiro, R., Aarrevaara, T., Berg, L. N., Fumasoli, T., Geschwind, L., Hansen, H. F., & Söderlind, J. (2019). Nordic higher education in flux: system evolution and reform trajectories. In R. Pinheiro, R. Pinheiro, L. Geschwind, H. Foss Hansen, & K. Pulkkinen (Eds.), Reforms, organizational change and performance in higher education a comparative account from the Nordic countries (1st ed. pp. 69–108). Springer Nature.
  • Pruvot, E. B., Claeys-Kulik, A.-L., & Estermann, T. (2015). Strategies for efficient funding of universities in europe. In A. Curaj, L. Matei, R. Pricopie, J. Salmi, & P. Scott (Eds.), The European higher education area. between critical reflections and future policies (pp. 153–168). Springer.
  • Ratner, H. (2020a). Europeanizing the Danish school through national testing: standardized assessment scales and the anticipation of risky populations. Science, Technology, & Human Values, 45(2), 212–234. https://doi.org/10.1177/0162243919835031
  • Ratner, H. (2020b). Topologies of organization: Space in continuous deformation. Organization Studies, 41(11), 1513–1530. https://doi.org/10.1177/0170840619874464
  • Rowlands, J., & Wright, S. (2021). Hunting for points: The effects of research assessment on research practice. Studies in Higher Education (Dorchester-On-Thames), 46(9), 1801–1815. https://doi.org/10.1080/03075079.2019.1706077
  • Saari, A. (2022). Topologies of desire: Fantasies and their symptoms in educational policy futures. European Educational Research Journal EERJ, 21(6), 883–899. https://doi.org/10.1177/1474904120988389
  • Slawomir, F. (2012). MULTI-YEAR BUDGETING–TOWARDS the ENHANCEMENT OFFISCAL DISCIPLINE. International Journal of Economics and Finance (Izmir), 4(1), 153–162. https://dergipark.org.tr/en/pub/ijefs/issue/26162/275593
  • Steunenberg, B. (2021). The politics within institutions for regulating public spending: Conditional compliance within multi-year budgets. Constitutional Political Economy, 32(1), 31–51. https://doi.org/10.1007/s10602-020-09323-5
  • Woelert, P., & McKenzie, L. (2018). Follow the money? how Australian universities replicate national performance-based funding mechanisms. Research Evaluation, 27(3), 184–195. https://doi.org/10.1093/reseval/rvy018
  • Zacharewicz, T., Lepori, B., Reale, E., & Jonkers, K. (2019). Performance-based research funding in EU member states—a comparative assessment. Science & Public Policy, 46(1), 105–115. https://doi.org/10.1093/scipol/scy041
  • Zerquera, D., & Ziskin, M. (2020). Implications of performance-based funding on equity-based missions in US higher education. Higher Education, 80(6), 1153–1174. https://doi.org/10.1007/s10734-020-00535-0