Fine-tuned convolutional neural nets for cardiac MRI acquisition plane recognition

Abstract

In this paper, we propose a convolutional neural network-based method to automatically retrieve missing or noisy cardiac acquisition plane information from magnetic resonance imaging and predict the five most common cardiac views. We fine-tune a convolutional neural network (CNN) initially trained on a large natural image recognition data-set (Imagenet ILSVRC2012) and transfer the learnt feature representations to cardiac view recognition. We contrast this approach with a previously introduced method using classification forests and an augmented set of image miniatures, with prediction using off the shelf CNN features, and with CNNs learnt from scratch. We validate this algorithm on two different cardiac studies with 200 patients and 15 healthy volunteers, respectively. We show that there is value in fine-tuning a model trained for natural images to transfer it to medical images. Our approach achieves an average F1 score of 97.66% and significantly improves the state-of-the-art of image-based cardiac view recognition. This is an important building block to organise and filter large collections of cardiac data prior to further analysis. It allows us to merge studies from multiple centres, to perform smarter image filtering, to select the most appropriate image processing algorithm, and to enhance visualisation of cardiac data-sets in content-based image retrieval.

Keywords::

• medical imaging and visualization
• applications of imaging and visualization
• data processing and analysis
• computer aided diagnosis
• image processing and analysis
• machine learning; cardiac magnetic resonance; convolutional neural networks

Acknowledgements

We used data and infrastructure made available through the Cardiac Atlas Project (www.cardiacatlas.org – Fonseca et al. (2011)). See Kadish et al. (2009) and Tobon-Gomez et al. (2013) for more details on the data-sets. A list of participating DETERMINE investigators can be found at http://www.clinicaltrials.gov. This work uses scikit-learn toolkit (Pedregosa et al. 2011) for decision forests and Caffe deep learning framework (Jia et al. 2014) for training of the convolutional neural network and the pretrained model (CaffeNet). This model was trained on a subset (Russakovsky et al. 2014) of the ImageNet (Deng et al. 2009) data-set.

Conflict of interest disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This work was supported by Microsoft Research through its Ph.D. Scholarship Programme and ERC Advanced Grant [MedYMA 2011-291080]. The research leading to these results has received funding from the European Unions Seventh Framework Programme for research, technological development and demonstration under [grant agreement no. 611823 (VP2HF)]. DETERMINE was supported by St Jude Medical, Inc. and the National Heart, Lung and Blood Institute [R01HL91069].