Segmentation of head-and-neck organs-at-risk in longitudinal CT scans combining deformable registrations and convolutional neural networks

ABSTRACT

Automated segmentation of organs-at-risk (OAR) in follow-up images of the patient acquired during the course of treatment could greatly facilitate adaptive treatment planning in radiotherapy. Instead of segmenting each image separately, the segmentation could be improved by making use of the additional information provided by longitudinal data of previously segmented images of the same patient. We propose a tool for automated segmentation of longitudinal data that combines deformable image registration (DIR) and convolutional neural networks (CNN). The segmentation propagated by DIR from a previous image onto the current image and the segmentation obtained by a separately trained cross-sectional CNN applied to the current image are given as input to a longitudinal CNN, together with the images itself, that is trained to optimally predict the manual ground truth segmentation using all available information. Despite the fairly limited amount of training data available in this study, an improvement of the segmentation of five different OAR in head-and-neck CT scans was found compared to both the results of DIR and the cross-sectional CNN separately in terms of Dice coefficient.

KEYWORDS:

• Segmentation
• convolutional neural networks
• deformable registration
• image processing
• organs-at-risk

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work is supported in part by Internal Funds KU Leuven under grant number C24/18/047. Siri Willems is supported by a Ph.D. fellowship of the Research Foundation – Flanders (FWO) under grant number 1SA6419N.

Notes on contributors

Liesbeth Vandewinckele

Liesbeth Vandewinckele is a PhD student at KU Leuven (Belgium) within the Laboratory of Experimental Radiotherapy. In 2018, she received her MSc in Biomedical Engineering and in 2019 her MSc in Medical Radiation Physics, both at KU Leuven. Her research is about the automation of radiotherapy treatment planning and is embedded within the Department of Radiation Oncology of the University Hospitals Leuven.

Siri Willems

Siri Willems is a MSc in Biomedical Engineering. Since 2018, she follows a PhD program at the KU Leuven within the Medical Imaging Research Center of the University Hospitals Leuven. Her main research focus is to develop learning-based strategies for automating the planning workflow in radiotherapy.

David Robben

David Robben received his MSc in Engineering in Computer Science from KU Leuven in 2012. After obtaining his PhD degree in 2016, he started as a postdoctorial researcher. Since 2018, he is research fellow at KU Leuven and researcher at icometrix, Leuven, Belgium. His research is about image processing, machine learning and computer vision in medical images.

Julie Van Der Veen

Julie van der Veen is radiation oncologist in training at the Department of Radiation Oncology at the University Hospitals Leuven, Belgium. After finishing Medical school in 2013, she started her training in Radiation Oncology. Since 2016 she is working on her PhD at KU Leuven in the field of head and neck cancer radiotherapy. Her research combines functional imaging with pathological investigation and delineation studies to improve quality of delineation and treatment delivery in head and neck cancer.

Wouter Crijns

Wouter Crijns obtained a Master in Theoretical Physics at the KU Leuven in 2006 and further specialized in Medical Radiation Physics at KU Leuven-UZ Leuven. He obtained a PhD degree in Biomedical sciences in 2015. He is certified by the Belgian Federal Agency of Nuclear Control (FANC) as Medical Radiation Physicist in Radiotherapy. Since 2015 he has worked as medical physics expert at the Department of Radiation Oncology, with a focus on treatment planning systems, 2D dosimetry. In 2018 he started as part-time assistant professor at the Department of Oncology, Laboratory of Experimental Radiotherapy of KU Leuven. His research interest is situated in automation and the use of prior knowledge in treatment planning by utilization of a.o. deep learning.

Sandra Nuyts

Sandra Nuyts is radiation oncologist and Full Professor at the Department of Radiation Oncology at KU Leuven, Belgium. After finishing Medical School and her training in radiation oncology, she defended her PhD thesis at KU Leuven in the field of tumour targeted gene therapy. Since 2002 she has been working at the Department of Radiation Oncology of University Hospitals Leuven, where she is responsible for the treatment and follow up of Head and Neck Cancer patients. She is leading a research group on translational research in the domain of head and neck cancer, focusing on prognostic and predictive markers, including HPV research and the use of functional imaging in radiotherapy.

Frederik Maes

Frederik Maes is professor at the Department of Electrical Engineering (ESAT) of KU Leuven and head of the division Processing Speech and Images (PSI). He received his MSc in Electrical Engineering from KU Leuven (Belgium) in 1991 and from Stanford University in 1992. After obtaining his PhD degree in Electrical Engineering from KU Leuven in 1998, he was appointed by KU Leuven as Assistant Professor in 2004 and promoted to Associate Professor in 2005 and Full Professor in 2018. His field of research is medical image computing. His research is strongly application-driven and embedded within the Medical Imaging Research Center of the university hospital UZ Leuven.