AMEE Guide No. 123 – How to read studies of educational costs

Abstract

Healthcare and health professions education share many of the same problems in decision making. In both cases, there is a finite amount of resources, and so choices need to be made between alternatives. To navigate the options available requires effective decision making. Choosing one option requires consideration of its opportunity cost – the benefit forgone of the other competing options. The purpose of this abridged AMEE guide is to introduce educational decision-makers to the economic concept of cost, and how to read studies about educational costs to inform effective cost-conscious decision-making. This guide leads with a brief review of study designs commonly utilized in this field of research, followed by an overview of how study findings are commonly presented. The tutorial will then offer a four-step model for appraising and considering the results of an economic evaluation. It asks the questions: (1) Can I trust the results? (2) What are the results telling me? (3) Could the results be transferred to my context? (4) Should I change my practice?

Glossary

Break-Even Analysis (BEA): An analytical approach that considers if and when return on investment may be expected, mapping outgoing costs (including startup costs) against incoming savings.

Break-Even point: The time-point at which incoming savings have balanced out the outgoing costs (including startup costs) - i.e. when return on investment has been achieved.

Cost-Analysis: An analytical approach that investigates costs alone, and not the subsequent returns or outcomes; often referred to as a partial economic analysis.

Cost-Benefit Analysis (CBA): An analytical approach that compares costs and benefits, with benefits (i.e., learning outcomes) expressed in monetary units (e.g. dollars).

Cost-Effectiveness Analysis (CEA): An analytical approach that compares costs and benefits, with benefits (i.e., learning outcomes) expressed in natural units i.e. knowledge change, or clinical competence.

Cost-Effectiveness Acceptability Curve (CEAC): A method of representing uncertainty in a study’s findings regarding willingness to pay.

Cost Minimisation Analyses (CMA): An analytical approach that provides information on which intervention or process has the least costs, assuming the outcomes of the interventions are equivalent.

Discounting: An analytical approach that gives greater weight to returns realized today relative to future returns, because of advantage of immediate use, as well as uncertainty and inflation.

Discount Rate: The numeric term used to discount future costs (see "Discounting"). The higher the discount rate the lower the value (weighting) of future dollars.

Dominance: A pattern of results showing that one intervention is both more effective and less costly than the alternative.

Economic Cost: The true cost of the intervention or process, including financial costs, as well as those of time and resources consumed.

Educational Decision-Maker: Anyone whose decisions can impact on learners, or teaching practices.

Incremental Costs: The additional cost in producing one additional unit of measure.

Modeled Costs: Cost estimates based on indirect approaches or external sources (as contrasted with actual costs, which have been specifically measured).

Net Present Value: An estimate of the financial value of all future benefits realized over a defined time period.

Opportunity Cost: The value of something, whether tangible or intangible, given up or sacrificed in order to obtain goods, services or inputs (i.e., the worth of this "something" if applied to its next best use).

Quality Assessed Students Educated (QASE): A measure of effect that takes into account the degree of change, and the quantity of students who experienced the change (QASE: number of students educated × the group’s average grade)(Maloney et al. 2015).

Perspective: The stakeholder viewpoint from which costs are being calculated i.e. are costs being considered from the perspective of the student, or perspective of the educational institution?

Sensitivity Analysis: An approach for representing uncertainty or variability in study findings, accomplished by repeating key analyses using alternative assumptions, conditions, or estimates.

Stakeholder Perspective: The stakeholder viewpoint of interest (e.g., the student, the health service, or the educational institution).

Start-up Costs: Costs incurred in initiating the activity, as distinct from ongoing or running costs e.g. infrastructure costs.

Sunk Costs: Costs that have been incurred prior to initiating an intervention/activity and that cannot be recovered.

Willingness to Pay (WTP): A measure of the stakeholders’ perceived value of the item, intervention or experience.

Acknowledgments

We would like to acknowledge the talented Melanie Crittle, for her original illustrations.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Additional information

Funding

The authorship team would like to acknowledge the generous funding received from the Office of the Vice-Provost (Learning and Teaching) of Monash University, that supported the pursuit and completion of this AMEE guide.

Notes on contributors

S. Maloney

Stephen Maloney, B.Physio, MPH, PhD, eMBA, is Director of Education for Monash University’s School of Primary and Allied Health Care, in the Faculty of Medicine, Nursing and Health Sciences. Stephen is a founding member of the International Society for Cost and Value in Health Professions Education, a fellow of the Monash Education Academy, and an adjunct of the Monash Education and Research Quality Unit.

D. A. Cook

David A. Cook, MD, MHPE, is Associate Director of Research in the Office of Applied Scholarship and Education Science, Chair of the Simulation Center Research Committee, and Professor of Medicine and Medical Education, Mayo Clinic College of Medicine; and a Consultant in the Division of General Internal Medicine, Mayo Clinic, Rochester, Minnesota, USA.

R. Golub

Robert Golub is the Deputy Editor, JAMA (Journal of the American Medical Association) and Associate Professor of Medicine, Northwestern University Feinberg School of Medicine Chicago, Illinois, USA.

J. Foo

Mr. Jonathan Foo, B.Physio, is a PhD candidate whose thesis is focused on the application of economic evaluations to health professions education. Jonathan is a practicing physiotherapist, Teaching Associate at Monash University Australia, and secretary of the International Society for Cost and Value in Health Professions Education.

J. Cleland

Professor Jennifer Cleland, BSc (Hons), MSc, PhD, D Clin Psychology is the John Simpson Chair of Medical Education Research at the University of Aberdeen, UK, where she leads the Centre for Healthcare Education Research and Innovation (CHERI). She is Chair of the Board of Management for Medical Education; Associate Editor for Perspectives in Medical Education; an Invited Member of the Wilson Centre, Toronto; Associate Professor at the Uniformed Services University of the USA; and Chair of the Association for the Study of Medical Education (ASME).

G. Rivers

Dr George Rivers, BEc(Hon); MEc; PhD is Associate Dean (Industry Research) with the Faculty of Business and Economics at Monash University. He is on the board of the Economic Society of Australia and is on the executive committee of the Society for Cost and Value in Health Professions Education. Dr Rivers has also been engaged by various government departments as a cost-benefit economist including the Department of Health and Human Services (Victoria).

M. G. Tolsgaard

Martin Tolsgaard, MD, PhD, DMSc. Associate Professor, University of Copenhagen and a medical education scientist at Copenhagen Academy for Medical Education and Simulation (CAMES). He is deputy editor for Advances in Health Sciences Education and for Perspectives on Medical Education.

D. Evans

Professor Darrell Evans, BSc, PhD is the Deputy Vice-Chancellor (Academic) at the University of Newcastle, Australia, Fellow of the Royal Society of Biology, the Royal Microscopical Society, the Higher Education Academy and the Anatomical Society. He is also an Honorary Fellow and Founder of the Monash Education Academy.

M. E. Abdalla

Mohamed Elhassan Abdalla, MB.BS, MHPE, PhD is an Assistant Professor of Medical Education and Director of Medical Education Unit, College of Medicine, University of Sharjah, UAE, and Chairperson for the Group on Social Accountability of Medical Schools at the Association of Medical Education in the Eastern Mediterranean Region (AMEEMR).

K. Walsh

Dr Kieran Walsh, MB, FRCPI is Clinical Director at BMJ. He is the editorial lead of the medical education, quality improvement and evidence based medicine products at BMJ. He is a Fellow of the Higher Education Academy, a Fellow of the Royal College of Physicians of Ireland and a Fellow of the Academy of Medical Educators. In the past he has worked as a hospital doctor specializing in General Internal Medicine and Geriatric Medicine.