A narrative review of course evaluation methods for continuing professional development: The case of paediatric and neonatal acute-care in-service courses in low and lower-middle income countries: BEME Guide No. 76

Abstract

Background

Training a skilled healthcare workforce is an essential part in reaching the United Nations Sustainable Development Goal to end preventable deaths in children and neonates. The greatest burden of mortality lies in low and lower-middle income countries (LLMIC). Short term, in-service courses have been implemented in many LLMIC to improve the quality of care delivered, but the evaluation methods of these courses are inconsistent.

Method

Studies describing evaluations of course and outcome measures were included if the course lasted seven days or less with postgraduate participants, included paediatric or neonatal acute or emergency training and was based in a LLMIC. This narrative review provides a detailed description of evaluation methods of course content, delivery and outcome measures based on ‘Context, Input, Process and Product’ (CIPP) and Kirkpatrick models.

Results

5265 titles were screened with 93 articles included after full-text review and quality assessment. Evaluation methods are described: context, input, process, participant satisfaction, change in learning, behaviour, health system infrastructure and patient outcomes.

Conclusions

Outcomes, including mortality and morbidity, are rightly considered the fundamental aim of acute-care courses in LLMIC. Course evaluation can be difficult, especially with low resources, but this review outlines what can be done to guide future course organisers in providing well-conducted courses with consistent outcome measures for maximum sustainable impact.

Keywords:

• Paediatric
• neonatal
• postgraduate
• education
• low-resource

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Acknowledgement

No funding was obtained for this review.

Author contributions

AG and MK conceived the review. AJ assisted AG in the database search strategies. AG, MK, RP, KP, HG, EH, JH, RI, AJ, JH, LB, RH, SW and JT were involved in the article screening and data extraction. AG led drafting of the manuscript with all authors providing critical revisions and approval of the final version. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Alison Gifford

Alison Gifford, BMBCh, MA MRes, DTMH, MRCPCH, is a Paediatric Specialty Trainee (ST5) and SCREDS Clinical Lecturer in Child Global Health at the University of Dundee. Her interests are in Paediatric Neurology including neurological infectious diseases and epilepsy. She is part of the Monitoring and Evaluation team for the Paediatric Epilepsy Training (PET) courses run by the British Paediatric Neurology Association which have trained over 14,000 participants over five continents. She has just completed a Clinical Research Fellowship and is due to start a MRC Clinical PhD in cryptococcal meningitis at the University of Exeter later this year.

Rune Philemon

Rune Philemon, MD, DTMH, MMed, is a Consultant Paediatrician from Tanzania working in Kilimanjaro, at the Kilimanjaro Christian Medical College. He has been involved in training medical and paramedical students at different levels for the past fifteen years. He is also involved in training and coordination postgraduate and various in-service trainings. He was the Tanzanian Lead for the East African Diploma of Tropical Medicine and Hygiene in collaboration with the London School of Hygiene and Tropical Medicine. He is passionate about improving health, particularly for children, and a strong believer that this can only be brought about by quality and continuous training.

Jay Halbert

Jay Halbert, MBBS, BSc, DTMH, is a Consultant Paediatrician currently practicing at Tunbridge Wells Hospital having previously been the Clinical Lead in Paediatric Oncology at Barts Health. He has worked as an expert advisor to the UK Healthcare Regulator (CQC), National Institute for Health and Care Excellence (NICE), Royal College of Paediatrics and Child Health (RCPCH) and to the World Health Organisation (WHO). Dr Halbert is a Principle Investigator for several national and international child health studies. Dr Halbert has a specialist interest in teaching and global child health with experience implementing child health programmes in Africa, Asia, Europe and Latin America. He is the founder of an Emergency Paediatric Care Programme in Myanmar through a partnership between UNICEF, RCPCH, the Myanmar Paediatric Society and Ministry of Health. Dr Halbert is an Advanced Paediatric Life Support Instructor and Medical Director and enjoys teaching on a variety of undergraduate and postgraduate courses including the Masters of Tropical Paediatrics and Diploma of Tropical Medicine in Liverpool.

Eleanor J. Hothersall

Eleanor J. Hothersall, BSc, MBChB, MPH, MMEd, MD, is Head of the MBChB programme at University of Dundee, Scotland, and Associate Director of Public Health at NHS Tayside. She is currently conducting a systematic review of assessment of highly contextual knowledge in undergraduate medical education. She is Co-chair of the Public Health Educators in Medical Schools group.

Rebecca Inglis

Rebecca Inglis, BMBCh, BA, DTMH, FFICM, is a British intensive care doctor and Fellow of the Faculty of Intensive Care Medicine. Her area of research is the delivery of Critical Care in low resource settings, with an emphasis on effective implementation. She set up the Essential Critical Care Course in collaboration with the Lao Critical Care Society. Dr Inglis is a founding member of CRIT CARE ASIA (the Collaboration for Research, Implementation and Training in intensive CARE in ASIA) which has established a regional network linking multiple intensive care units across South and Southeast Asia. She has worked for Médecins Sans Frontières and taught on the Diploma for Tropical Medicine and Hygiene. During the pandemic Rebecca has been working as a consultant for the World Health Organization, supporting the COVID response in Lao PDR.

Jo Hart

Jo Hart, BSc(Hons), MSc, PhD, CPsychol, PFHEA, is a Health Psychologist and Professor of Health Professional Education based at the University of Manchester, UK where she is Head of the Division of Medical Education. She studies and implements interventions focused on education and training of healthcare professionals. Prof Hart is health professional education lead for The Change Exchange (https://thechangeexchange.org), in which health psychologists volunteer to work with health partnerships between UK and low income country healthcare organisations. She works with Health Education England and the UK Health Security Agency, influencing the use of behavioural science in education and training. She is a Past Chair of the British Psychological Society Division of Health Psychology and is interested in the development of health psychology in the UK and globally.

Lucie Byrne-Davis

Lucie Byrne-Davis, BSc(Hons), MSc, PhD, is a professor of Health Psychology at the University of Manchester and an HCPC registered Health Psychologist. Her research and practice focus on translating health psychology to support behaviour change, particularly in health worker behaviours. She is responsible for the behavioural and social sciences education of the undergraduate medical students at University of Manchester. She is a Fellow of the Academy of Social Sciences, a Principal Fellowship of the Higher Education Academy and a Fellow of the European Health Psychology Society, where she is Chair of the UN committee. She is the Chair of the Division of Health Psychology of the British Psychological Society (2021–2023).

Joanna Thirsk

Joanna Thirsk, MBChB, MSc, FRCA, FFICM, DCH, DA DipPEC, completed her MBChB at the University of Cape Town in South Africa in 2005. After overwintering as part of the 48th South African National Antarctic expedition she undertook a Clinical Research Fellowship in Intensive Care at Kings College Hospital in London and subsequently dual trained in Anaesthesia and Intensive Care in Aberdeen, Scotland. She completed a Masters in Paediatric Emergency Medicine at the University of Edinburgh and has a Diploma in Child Health. She is currently sub-specialising in paediatric intensive care medicine at University Hospital Southampton. Dr Thirsk has a strong interest in global and expedition medicine especially in the areas of child and maternal health. She is also a proud SAFE (Safer Anaesthesia From Education) instructor and has taught in LMICs from Somaliland to India. Dr Thirsk holds Diplomas in Primary Emergency Care (DipPEC) and Anaesthesia (DA) from the Colleges of Medicine of South Africa. She is also part of the social media team for the online educational platform Don’t Forget the Bubbles. Dr Thirsk is a fellow of both the Royal College of Anaesthetists and the Faculty of Intensive Care Medicine. In her spare time, Joanna enjoys travelling especially to the polar and sub polar regions.

Hugh Gifford

Hugh Gifford, BMBCh, MA MRes, DTMH, MRCP (UK), FFICM, is an Intensive Care & Respiratory Medicine senior registrar (ST7) in the East of Scotland, currently researching a PhD in the global public health threat and emerging human fungal pathogen Candida auris on an MRC scholarship at the University of Exeter Center for Medial Mycology.

Rachel Howells

Rachel Howells, MBBChir, MRCPCH, Dip Med Ed, is an acute paediatrician at the Royal Devon University Healthcare NHS Foundation Trust, specialising in Paediatric Neurology. As a regular teacher of short courses in epilepsy and headache for professionals in the UK and abroad, Dr Howells is interested in how good short courses are in changing the effectiveness of healthcare provision and outcomes for children.

Shona Weetch

Shona Weetch, MA(Hon), MBChB, PGCert, is a Clinical Fellow in Intensive Care Medicine in Glasgow, Scotland. Prior to studying Medicine she gained an MA (Hons) in Spanish, and maintains a great interest in travel and global health. She has studied in Peru and Colombia and is currently undertaking a PG Diploma in Healthcare Professions Education at the University of Glasgow. She will start ACCS Anaesthetics training this year.

Katie Prentice

Katie Prentice, MBChB, BMSc, MRCPCH, is a Paediatric Specialty Trainee (ST7) with a specialist interest in high dependency care working in Paediatric Critical Care Unit in the Royal Hospital for Children and Young People in Edinburgh.

Andy Jackson

Andy Jackson, BA(Hons), HNC, PGCertHE, is the Learning and Teaching Librarian at the University of Dundee and the University’s Medical Librarian. He regularly teaches on undergraduate and postgraduate programmes for Medicine, Dentistry and Nursing. His professional interests include the role of information in evidence-based practice, teaching skills for library staff and digital literacy as a tool for employability. He has also written and edited several books of poetry.

Martin Kirkpatrick

Martin Kirkpatrick, MBBS, FRCPCH, FRCPE, DCH, holds the appointments of Consultant Paediatric Neurologist at Tayside Children’s Hospital, Dundee and Honorary Professor of Child Health in the School of Medicine at the University of Dundee. Martin was the founder Lead Clinician of the North Scotland Child and Adolescent Neurology Network, set up with the aim of providing equitable specialist paediatric neurology care across widely geographically dispersed populations. He is also a founder member of the formally commissioned Scottish Paediatric Epilepsy Network and, through RCPCH, was instrumental in making the case for and setting up the “Epilepsy12” UK national audits. Martin’s research interests are in medication adherence in epilepsy and in epilepsy clinical service design and audit. He is a current member of the International League Against Epilepsy (ILAE) Standards & Best Practice Council and is Chair of the ILAE Implementation and Education Taskforce. Martin has had a longstanding interest in international development and has worked in Nepal, Iraq, Rwanda and Kosovo. He has helped to develop the international component of British Paediatric Neurology Association short-course education that has now expanded across five continents. At present, he is Chair of International Education at the BPNA.