International comparability of reference unit costs of education services: when harmonizing methodology is not enough (PECUNIA project)

ABSTRACT

Background

Health problems can lead to costs in the education sector. However, these costs are rarely incorporated in health economic evaluations due to the lack of reference unit costs (RUCs), cost per unit of service, of education services and of validated methods to obtain them. In this study, a standardized unit cost calculation tool developed in the PECUNIA project, the PECUNIA RUC Template for services, was applied to calculate the RUCs of selected education services in five European countries.

Methods

The RUCs of special education services and of educational therapy were calculated using the information collected via an exploratory gray literature search and contact with service providers.

Results

The RUCs of special education services ranged from €55 to €189 per school day. The RUCs of educational therapy ranged from €6 to €25 per contact and from €5 to €35 per day. Variation was observed in the type of input data and measurement unit, among other.

Discussion

The tool helped reduce variability in the RUCs related to costing methodology and gain insights into other aspects that contribute to the variability (e.g. data availability). Further research and efforts to generate high quality input data are required to reduce the variability of the RUCs.

KEYWORDS:

• Economic evaluation
• intersectoral costs and benefits
• PECUNIA
• reference unit costs
• valuation

1. Introduction

Health and education are interrelated. On the one hand, education is one of the social determinants of health, which is associated with better health outcomes [1–3]. This association can be explained by the healthier lifestyle and higher health literacy of people with a higher education, along with other mechanisms [1–3]. On the other hand, a person’s ability to participate in education can be affected by their health status [4]. Poor health has been associated with lower academic achievement [5] and lower educational attainment [6]. Accordingly, interventions with the primary intent of improving health also have the potential to improve educational outcomes. For example, Belot and James [7] find that healthy school meals were associated with higher test scores and lower rates of absenteeism. Similarly, the findings of a study by Levine and Schanzenbach [8] suggest that health insurance coverage was associated with improved test scores in reading.

Health problems also often have economic implications for the education sector, particularly in the population of children and adolescents. For example, the study by Le and colleagues [9] has shown that costs associated with providing additional education for children and adolescents with attention deficit hyperactivity disorder could account for up to 60% of the total costs associated with the condition. At the same time, health interventions can lead to economic benefits in the education sector, as demonstrated in the study by Wellander and colleagues [10], using the example of an intervention to reduce mental health problems among students. Accordingly, taking education costs into account in health economics research could be crucial for informing optimal decision-making at the societal level, particularly in the context of interventions targeting children and adolescents with health problems.

Yet the number of health economics studies that take into account costs in the education sector is limited [11,12]. The predominance of the narrow healthcare perspective in the majority of economic evaluations performed to date [11,12] can be attributed to the specific requirements of the national health technology assessment bodies in many countries (e.g. in the United Kingdom and Belgium) [13], the lack of clear recommendations and consensus on what a societal perspective should entail [12], but also to the lack of appropriate methods and tools to support the inclusion of costs outside of the healthcare sector in economic evaluations. The lack of valid unit costs of education services as well as the limited methodological guidance on how to properly value education costs in health economics studies are example areas of where the development of methods and tools is urgently needed. As demonstrated in the recent review, studies in which education costs and/or consequences were taken into account vary widely, in particular in terms of the valuation methods [14]. For example, in two recent studies in Sweden, the costs of additional educational support were valued differently: Lenhard et al. [15] based their estimate on national-level secondary data and Jolstedt et al. [16] used the average market price of this service. Moreover, many studies did not report how the education costs and/or consequences were valued, which further hampers the comparability between the estimates and hence the study results [14]. To improve the comparability of health economics cost data, several countries developed national unit costs for common components of resource-use (e.g. health and social care services, medication prices) [17,18]. However, the unit costs of education services are rarely available. Furthermore, while national unit costs facilitate the comparability of national evidence, the cross-country comparability of unit costs remains an issue. This is particularly relevant given the growing emphasis on cooperation and joint health technology assessment initiatives in the European Union [19].

The standardization of valuation methods and the development of multi-national unit costs in health economics research has the potential to improve the comparability, applicability, and transferability of health economic evidence within and across countries. Therefore, the ProgrammE in Costing, resource use measurement, and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA) project set out to develop standardized methods and tools for resource-use measurement and unit cost calculation, with the aim of producing comparable cost data for health economic evaluations across Europe [20]. PECUNIA also broadened the scope beyond the healthcare sector and incorporated other societal sectors that can be relevant for health economic evaluations: health and social care, education, criminal justice, labor and productivity, and patient, family and informal care. This particular study focuses on the education sector. The aim of this study is to provide the reference unit costs (RUCs) for selected educational services by applying the standardized unit cost calculation tool, the PECUNUA RUC template for services, in five European countries. These RUCs are available for use in future national and multi-national economic evaluations in Europe in an online database, the PECUNIA RUC Compendium [21].

2. Methods

2.1. PECUNIA RUC template for services

The study aimed to calculate and compare the RUCs of selected services in the education sector by applying the PECUNIA RUC template, a methodologically validated Excel template for a guided stepwise calculation of RUCs [22]. The PECUNIA RUC template for services is a Microsoft Excel-based tool, which allows for developing unit costs using either a top-down micro-costing or a top-down gross costing approach depending on the available data. Both approaches entail average unit cost estimation based on comprehensive data (e.g. annual accounts of an organization) [23]. Micro-costing refers to the identification of cost components in detail, while in gross costing cost components are identified at an aggregate level [23,24]. The template can be completed using either primary or secondary input data. Complementary data collection sheets were developed to facilitate the collection of primary data. A more detailed description of the template and its development is available elsewhere [22,25]. In this study, the template for the RUC calculation of services using top-down gross costing was used. This template was validated by the PECUNIA group by calculating RUCs of various services, including one service provided in the education sector in five of the six countries that participated in the PECUNIA project (Austria, Germany, Hungary, the Netherlands, and the United Kingdom (UK)) in November 2019.

2.2. Selection of services

For this study, the template for calculating the RUCs of services was applied to three selected education services in Austria, Germany, Hungary, the Netherlands, and the UK. The services were selected from a comprehensive overview of costs and consequences in the education sector, developed at an earlier stage of the PECUNIA project. The list was developed by conducting a systematic review of peer-reviewed and gray literature and a survey completed by an international group of experts in the education sector. For a detailed description of the development process of the list of services, we refer the reader to Pokhilenko et al. [26]. From this list, three education services were selected: ‘education services provided in a special education school (either primary or secondary),’ ‘educational therapy provided in primary schools,’ and ‘educational therapy provided in secondary schools.’ Based on previous research, it was expected that these three services would exist in all participating PECUNIA countries and be frequently used by children and adolescents with mental health problems [14]. Furthermore, each of the services was accompanied by a more detailed description developed by the authors of the study in consultation with the expert in the education sector (Table 1).

Table 1. Education services and service descriptions selected for reference unit cost calculation.

Service name	Service description
Education services provided in a special education school (either primary or secondary)	Education services provided in the special education school, as a day care facility, for students with mental or physical disabilities
Educational therapy provided in primary schools	Additional educational support provided at a regular primary school to students with learning difficulties (often referred to as remedial teaching)
Educational therapy provided in secondary schools	Additional educational support provided at a regular secondary school to students with learning difficulties (often referred to as remedial teaching)

2.3. Data collection

For this study, the top-down gross costing approach was selected by the PECUNIA group as it was the only method feasible given the timeframe of the project and the ongoing COVID19 pandemic. Top-down gross costing relies on aggregated data gathered, for example, in the reports of national ministries or national statistics organizations [27]. To collect relevant input data, an exploratory search of gray literature (e.g. ministry reports, and reports of national statistics organizations) was conducted in all five countries that participated in this study, namely Austria, Germany, Hungary, the Netherlands, and the UK. The minimally required data for calculating the RUCs using this approach included total costs associated with the provision of each service for a given period of time (i.e. per year) and the total number of client contacts for the same period of time. For insights into the comparability of the estimates, additional information regarding the components of the RUCs (e.g. staff costs, overhead costs) was required: in case the source did not include information in the required level of detail, the authors contacted the data provider directly. Any additional assumptions necessary for calculating the RUCs (e.g. regarding the average number of school days per year or the average number of students per classroom) were transparently documented and motivated by empirical data where possible. In case no data were available to develop the RUCs for the pre-defined service(s), proxy services were identified by the coauthors in each country based on data availability and proximity to the original service description.

In case no secondary data were available, the alternative strategy was to collect primary data. Relevant service providers, i.e. the providers of the service for which the RUC was developed, were contacted and asked to fill out a separate data collection sheet. In the data collection sheet, the service providers were asked to provide information regarding the costs incurred due to providing the given service, such as annual direct and overhead costs and the number of client contacts for a given period of time (e.g. in a given year). These data were then fed into the PECUNIA RUC template.

The RUCs of the three selected education services were compared based on several aspects, including the measurement unit, costing method and the type of input data used for the calculation, components included in the unit cost, and the representativeness of the estimates. To facilitate the comparison, all values were converted to 2019 Euros using the national Consumer Price Index.

3. Results

3.1. Education services provided in a special education school

The RUCs of the education services provided in a special education school ranged from €54.99 per school day in the Netherlands to €188.85 per school day in Austria (Table 2). All RUCs corresponded to the measurement unit of ‘school day per student’ and were calculated based on secondary data. The Dutch government report used for the RUC calculation allowed for the calculation of separate estimates for the education services provided in primary and secondary special education schools [28]. The RUC of the services provided in secondary special education schools was approximately twice as high as the RUC of the services provided in primary special education schools. Four out of five RUCs were national-level estimates; the RUC in the UK was a local authority level estimate.

Table 2. Reference unit costs of education services provided in a special education school (2019 EURO).

Country	Reference unit cost (EUR 2019)	Measurement unit	Input data	Representativeness	Cost components included
Austria	188.85	School day per student	Secondary data	National	Staff costs, investments, overhead costs
Germany	99.54	School day per student	Secondary data	National	Staff costs, overhead costs, additional budget from the federal governments allocated to schools to support education of students with special needs
Hungary	64.02	School day per student	Secondary data	National	Staff costs, other direct costs, overhead costs
The Netherlands	54.99^1	School day per student	Secondary data	National	Average expenditure per student excluding the cost of housing
	124.53^2	School day per student	Secondary data	National	Average expenditure per student excluding the cost of housing
The United Kingdom	73.12	School day per student	Secondary data	Local authority	Local authority special education needs funding

¹education services provided in a primary special education school²education services provided in a secondary special education school

3.2. Educational therapy provided in primary schools

The RUCs for the educational therapy provided in primary schools ranged from €7.73 to €25.42 per contact per student and from €4.51 to €34.56 per school day per student (Table 3). In Austria no data on educational therapy were available. Therefore, the RUC was based on the proxy service – education services provided in a regular primary school. Since approximately 61% of students with special needs attend regular primary and secondary schools [29], this service was considered to be the closest alternative given the absence of appropriate data. The RUC was based on the average staff costs per hour of teaching per student in a regular primary school. Similarly, in Germany the majority of the students with special needs attend regular primary schools. The RUC of educational therapy was based on the average cost per hour of teaching per student in a regular primary school. In Hungary, the RUC of educational therapy was calculated as the difference in daily costs between schools with and without programmes for students with special education. Furthermore, since in Hungary primary and secondary education is typically provided in the same institution, a single estimate was calculated for educational therapy provided in primary and secondary schools. In the Netherlands, the RUC was calculated for a proxy service – remedial teaching provided in a private setting after school instead of in a daycare facility (as in the original definition). This estimate was based on the primary data collected from the service provider and it was the same regardless of the education level, i.e. the cost was the same for primary and secondary school students.

Table 3. Reference unit costs of educational therapy provided in a primary school (2019 EURO).

Country	Reference unit cost (EUR 2019)	Measurement unit	Input data	Representativeness	Cost components included
Austria	7.73	Contact per student	Secondary data	National	Staff costs
Germany	34.56	School day per student	Secondary data	National	Staff costs, overhead costs, additional budget from the federal governments allocated to schools to support education of students with special needs
Hungary	4.51^1	School day per student	Secondary data	National	Expenditure on teachers’ salaries, expenditure on other salaries, material costs, investment, renovation, other expenditure
The Netherlands	25.42^1	Contact per student	Primary data	Regional	Staff costs, costs for purchasing equipment
The United Kingdom	23.99	School day per student	Secondary data	Local authority	Local authority special education needs funding

¹The RUCs for educational therapy provided in primary and secondary schools are the same.

3.3. Educational therapy provided in secondary schools

The RUCs of educational therapy provided in secondary schools ranged from €5.82 to €25.42 per contact per student and from €4.51 to €13.36 per school day per student (Table 4). The calculation of the RUCs for Austria and Germany followed the same principle as described in paragraph 3.2 using the data on regular secondary schools. The Hungarian and Dutch RUCs for this service were the same regardless of the educational level.

Table 4. Reference unit costs of educational therapy provided in a secondary school (2019 EURO).

Country	Reference unit cost (EUR)	Measurement unit	Input data	Representativeness	Cost components included
Austria	5.82	Contact per student	Secondary data	National	Staff costs
Germany	14.64	Contact per student	Secondary data	National	Staff costs, overhead costs, additional budget from the federal governments allocated to schools to support education of students with special needs
Hungary	4.51^1	School day per student	Secondary data	National	Expenditure on teachers’ salaries, expenditure on other salaries, material costs, investment, renovation, other expenditure
The Netherlands	25.42^1	Contact per student	Primary data	Regional	Staff costs, costs for purchasing equipment
The United Kingdom	13.36	School day per student	Secondary data	Local authority	Local authority special education needs funding

¹The RUCs for educational therapy provided in primary and secondary schools are the same.

3.4. Differences and similarities of RUCs

The RUCs corresponded to either of the two harmonized units of measurement: ‘per school day’ (n = 10, 67%) or ‘per contact’ (n = 4, 33%). Ten RUCs were nationally representative, one RUC was representative of a specific region, and three RUCs were representative of the local authority level. The majority of the RUCs (n = 13, 87%) were calculated based on publicly available secondary data; one RUC was calculated based on primary data obtained from the service provider. The RUCs varied in terms of the cost components included. The most frequently included cost components included staff costs (n = 11) and overhead costs (n = 6). Furthermore, there were differences in the definition of the services for which the RUCs were developed. For example, the Dutch RUC of educational therapy was calculated based on the proxy service of remedial teaching provided in the private setting. Further details of the RUCs can be obtained from the PECUNIA RUC Compendium [21] or from the Supplementary File 1 which outlines the details of the RUC calculation.

4. Discussion

This study applied the novel standardized tool for unit cost calculation developed in the PECUNIA project, the PECUNIA RUC Template for services, to three selected education services in Austria, Germany, Hungary, the Netherlands, and the UK. The RUCs calculated in this study could be used by researchers in future economic evaluations, although the remaining limitations of the estimates need to be taken into account. Therefore, when including the RUCs listed in the current study in a health economic analysis, we recommended conducting a sensitivity analysis to explore the extent of the impact of the selected unit costs on the study results.

The results also suggest that despite an attempt to standardize the methodological approach, many other potential sources of variability affect the comparability of the RUCs – for example differences in the definition of the services and in the type of data available for the calculation. The PECUNIA RUC template for services has the potential to facilitate the comparability of the RUCs by offering a standardized and transparent methodological approach to RUC calculation. However, researchers should be aware that other aspects beyond methodology, such as those related to data availability and composition, can limit the comparability of the RUCs and require further international standardization and harmonization of input data collection methods. For example, if the components of RUCs are different (e.g. overhead costs are (not) included), the estimates cannot be directly compared.

The absence of a universally accepted gold standard for the valuation of services in health economics might lead to variability in the RUCs. For example, Mayer and colleagues calculated a unit cost of general practitioner consultation in Austria using six costing methods, and found an overall variation of 173% [30]. Despite the use of a standardized methodological approach, the results of this study still demonstrate substantial variation. For example, the RUCs of education services provided in a special education school ranged from €55 to €189. The application of the PECUNIA RUC template for the calculation of multi-national unit costs of health and social services shows similar levels of variation and suggests that the problem of variability is not unique to the education sector [25].

Variability of RUCs can lead to lack of comparability of the results of economic evaluations and can affect the decisions made based on the results. Variability can be attributed to various reasons. Mogyorosy and Smith [27] outline eight potential sources of variability of RUCs of health services, including differences in costing methodologies (e.g. differences in resource use measurement and valuation), differences in cost accounting systems and practices, differences in inclusion and exclusion of particular costs, differences in controlling regional and seasonal variations during costing, structural/organizational differences between providers, differences in patient case-mix and socio-demographic characteristics, and differences in financial and non-financial incentive systems, as well as geographical differences in input prices.

In this study, several potential explanations for the variability could be observed. First, the description of services differed from the originally selected prototype services across the five countries. This can be attributed to many factors, including differences in national educational systems, differences in the interpretation of the services and their descriptions, and the lack of (suitable) input data to calculate the RUC for the originally selected service. Second, differences in the type of data used for the calculation could contribute to the variation. When using aggregate secondary data for top-down gross costing or relying on existing estimates, it is not always clear what components are included in the calculation. On the other hand, calculating the RUCs on the basis of primary data may produce an estimate that is only relevant for a specific context and is not directly transferable. Differences in the type of input data available was also one of the major observed sources of variability related to the differences in cost accounting practices. This was reflected in the differences in the measurement unit, the representativeness of the data and the components included in the data. When available data were limited, the researchers had to make assumptions relying on the published literature but also on their own judgment, which further limited the comparability of the estimates. Geographical differences in input prices could also potentially explain some of the variability, which is not related to the costing methodology. However, the RUCs calculated in this study were not adjusted for purchasing power parity to retain their validity within local contexts.

4.1. Methodological reflection

This study presents the first attempt to apply a standardized and harmonized tool, the PECUNIA RUC template for services, to the calculation of the RUCs of education services in health economics. The tool was tested in five European countries and it provides insights into different sources of variability of the unit costs. It is important to note that while the PECUNIA RUC Template provided a structured methodological approach to the calculation of RUCs, the input data for the calculation needed to be obtained from various primary or secondary sources. Therefore, the quality of the RUCs remains dependent on the quality of the input data used for the calculation. The RUCs calculated based on reliable nationally representative secondary data are expected to be of higher quality in comparison with the RUCs calculated based on primary data collected from a single service provider (e.g. the RUC of educational therapy in the Netherlands).

4.2. Setting research agenda

The current study and the overall PECUNIA Project were prompted by the lack of harmonization, standardization and overall guidance on how to approach unit cost calculation in health economics research. This is important because a standardized approach can help produce comparable health economics data for informing policy decision-making. Although the PECUNIA RUC template for services provides an important contribution to standardizing unit cost calculation in health economics, there are other research gaps that could guide future research in this domain. First, focusing on the aspects that can affect variability of the RUCs proposed by Mogyorosy and Smith [27], further insight is needed into the aspects that could not be detected in the current study. Also, shedding light on the aspects that affect the variability the most can help concentrate efforts on optimizing the approach to unit cost calculation more effectively. Second, variation was observed in the description of services across countries. The use of a standardized terminology and classification system can help facilitate national and international comparison of services. One example of such a system that has been used in health economics is the standard taxonomy for description, mapping and comparison of services for long-term care, DESDE-LTC [31]. DESDE-LTC was developed to classify services in different societal sectors, including services in the education sector. This system provides common terminology and codes to enable comparison of services across settings. The use of DESDE-LTC for the classification of services in the education sector in health economics research has been explored in the PECUNIA project; however, further work to develop and test the classification of services in the education sector is needed to ensure that the system is fully functional.

Third, the results of the current study clearly demonstrate that both the availability and the type of data available for unit cost calculation are major contributors to unit cost variability. This highlights the need for promoting uniform accounting and data reporting standards across Europe. Fourth, as available RUCs for education services are limited, establishing a RUC database that includes education costs among costs in other sectors could facilitate the inclusion of intersectoral costs and consequences in health economics studies. This was tackled in the PECUNIA project by developing the PECUNIA RUC Compendium. The PECUNIA RUC Compendium is a multi-sectoral, multi-country database of RUCs comparable across countries and sectors developed using standardized and scientifically validated methods [21]. It is designed to be a living database planned to be updated and expanded both in terms of the services covered and countries. Finally, as unit cost calculation in health economics often involves a trade-off between accuracy and practicality, transparent reporting of the methodological approach is crucial for correct interpretation. This can be facilitated by the use of the PECUNIA costing tools – the PECUNIA RUC Templates and the PECUNIA RUC Compendium.

5. Conclusion

This study presents the first standardized calculation of RUCs of education services using the newly developed tool, the PECUNIA RUC Template for services, for use in health economic evaluations across Europe. The RUC presented in this study can be used by health economists who are undertaking economic evaluations, for which costs in the education sector would be of relevance. However, while the use of a standardized costing tool helped harmonize the methodological approach toward unit cost calculation, the RUCs still varied, indicating that differences in the RUCs can be attributed to aspects beyond methodology (e.g. the type of input data used for the calculation). The PECUNIA RUC Template for services can help reduce variability attributed to the methodological approach, but further harmonization is required to tackle other potential sources of variability such as input data availability. The insights gained from this study can also be of interest to European policy advisors by helping them strengthen methodological competence when interpreting results of multi-national health economic studies. Furthermore, this research can help facilitate the development of multi-sectoral funding arrangements for healthcare interventions that can lead to benefits in other sectors, for example, by reducing the need for special education among students with mental problems.

Declaration of interest

The author(s) declare that they are partially funded by the PECUNIA Project. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

All authors were involved in the conception and design of the study, as well as the analysis and interpretation of the data. The manuscript was drafted by I Pokhilenko and T Kast. All co-authors were involved in revising the manuscript critically for intellectual content and gave the final approval of the version to be published. All authors agreed to be accountable for all aspects of the work.

Acknowledgments

This manuscript was written on behalf of the PECUNIA group. Members of the PECUNIA Group are:

Medical University of Vienna: PI: Judit Simon; team members (in alphabetical order): Michael Berger, Claudia Fischer, Agata Łaszewska, Susanne Mayer, Nataša Perić;

University Medical Center Hamburg-Eppendorf: PI: Hans-Helmut König; team members (in alphabetical order): Christian Brettschneider, Marie Christine Duval, Paul Hinck, Johanna Katharina Hohls, Alexander Konnopka, Louisa-Kristin Muntendorf;

Corvinus University of Budapest: PI: Valentin Brodszky; team members: László Gulácsi;

Maastricht University: PI: Silvia M.A.A. Evers; team members (in alphabetical order): Ruben M.W.A. Drost, Luca M.M. Janssen, Aggie T.G. Paulus, Irina Pokhilenko;

Erasmus University Rotterdam: PI: Leona Hakkaart-van Roijen; team members (in alphabetical order): Kimberley Hubens, Ayesha Sajjad;

Servicio de Evaluación del Canario de la Salud: PI: Pedro Serrano-Aguilar; team members (in alphabetical order): Lidia García-Pérez, Renata Linertová, Lilisbeth Perestelo-Pérez, Cristina Valcárcel-Nazco;

PSICOST Scientific Association: PI: Luis Salvador-Carulla; team members (in alphabetical order):

Nerea Almeda, Pilar Campoy-Muñoz, Carlos R. García-Alonso, Mencía R. Gutiérrez-Colosía, Cristina Romero-López-Alberca;

London School of Economics and Political Science: PI: A-La Park;

University of Bristol: PI: William Hollingworth; team members (in alphabetical order): Sian Noble, Joanna Thorn.

This manuscript was presented and discussed at the lowlands Health Economics Study Group annual conference (lolaHESG) on 27 May 2021 and at the International Health Economics Association (iHEA) conference held in July 12-15, 2021. The co-authors would like to thank the discussant Dr. Brigitte Essers and all others who were present and contributed to the discussion.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14737167.2023.2152331

Additional information

Funding

This study was conducted alongside the ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA), which has received funding from the European Union’s Horizon 2020 research and innovation programme.