Entropy-based deep neural network training optimization for optical coherence tomography imaging

References

• Alqudah, A. M. 2020. AOCT-NET: A convolutional network automated classification of multiclass retinal diseases using spectral-domain optical coherence tomography images. Medical & Biological Engineering & Computing 58 (1):41–25. doi:10.1007/s11517-019-02066-y
   PubMed Web of Science ®Google Scholar
• Altan, G. 2022. DeepOCT: An explainable deep learning architecture to analyze macular edema on OCT images. Engineering Science and Technology, an International Journal 34:101091. doi:10.1016/j.jestch.2021.101091
   Google Scholar
• Apostolopoulos, S., C. Ciller, S. De Zanet, S. Wolf, and R. Sznitman. 2017. RetiNet: Automatic AMD identification in OCT volumetric data. Investigative Ophthalmology & Visual Science 58 (8):387–387.
   Web of Science ®Google Scholar
• Bai, Y., E. Yang, B. Han, Y. Yang, J. Li, Y. Mao, G. Niu, and T. Liu. 2021. Understanding and improving early stopping for learning with noisy labels. Advances in Neural Information Processing Systems 34:24392–403.
   Google Scholar
• Bonet, D., A. Ortega, J. Ruiz-Hidalgo, and S. Shekkizhar. 2021. Channel-wise early stopping without a validation set via NNK polytope interpolation. arXiv preprint arXiv:2107.12972.
   Google Scholar
• Brownlee, J. 2020. Understand the impact of learning rate on neural network performance. Accessed September 3 2021. https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/
   Google Scholar
• Bu, Y., S. Zou, and V. V. Veeravalli. 2020. Tightening mutual information-based bounds on generalization error. IEEE Journal on Selected Areas in Information Theory 1 (1):121–30. doi:10.1109/JSAIT.2020.2991139
   Google Scholar
• Duvenaud, D., D. Maclaurin, and R. Adams. 2016. Early Stopping as Nonparametric Variational Inference. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, ed. A. Gretton and C. C. Robert, 1070–1077. Cadiz, Spain: Proceedings of Machine Learning Research (PMLR).
   Google Scholar
• Forouzesh, M., and P. Thiran. 2021. “Disparity between batches as a signal for early stopping.” In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II 21, 217–32. Springer.
   Google Scholar
• Gu, J., Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai, et al. 2018. Recent advances in convolutional neural networks. Pattern Recognition 77:354–77. doi:10.1016/j.patcog.2017.10.013
   Web of Science ®Google Scholar
• He, K., X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, June 26 - July 1, Las Vegas, Nevada, 770–78.
   Google Scholar
• Hoffman, J. I. E. 2019. Chapter 25 - analysis of variance. I. One-way. In Basic biostatistics for medical and biomedical practitioners, ed. J. I. E. Hoffman, 391–417. 2nd ed. London: Academic Press.
   Google Scholar
• Irsch, K. 2021. Optical principles of OCT. In Albert and Jakobiec’s principles and practice of ophthalmology, ed. Daniel M. Albert, Joan W. Miller, Dimitri T. Azar and Lucy H. Young, 1–14. Springer Nature Switzerland AG: Springer.
   Google Scholar
• Janocha, K., and W. M. Czarnecki. 2017. On loss functions for deep neural networks in classification. CoRR abs/1702.05659. http://arxiv.org/abs/1702.05659
   Google Scholar
• Karthik, K., and M. Mahadevappa. 2023. Convolution neural networks for optical coherence tomography (OCT) image classification. Biomedical Signal Processing and Control 79:104176. doi:10.1016/j.bspc.2022.104176
   Web of Science ®Google Scholar
• Kermany, D. S., M. Goldbaum, W. Cai, C. C. Valentim, H. Liang, S. L. Baxter, A. McKeown, G. Yang, X. Wu, F. Yan, et al. 2018. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–31.e9. doi:10.1016/j.cell.2018.02.010
   PubMed Web of Science ®Google Scholar
• Kermany, D., K. Zhang, and M. Goldbaum. 2018. Labeled optical coherence tomography (OCT) and chest X-ray images for classification (2018). Mendeley Data V2. doi:10.17632/rscbjbr9sj.2
   Google Scholar
• Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25: 1097–105.
   Google Scholar
• Lindsay, G. W. 2021. Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of Cognitive Neuroscience 33 (10):2017–31. doi:10.1162/jocn_a_01544
   PubMed Web of Science ®Google Scholar
• Liu, K., R. Ali Amjad, and B. C. Geiger. 2018. Understanding individual neuron importance using information theory. arXiv Preprint arXiv: 1804.06679.
   Google Scholar
• Liu, Y., J. A. Starzyk, and Z. Zhu. 2008. Optimized approximation algorithm in neural networks without overfitting. IEEE Transactions on Neural Networks 19 (6):983–95. doi:10.1109/TNN.2007.915114
   PubMedGoogle Scholar
• Li, T., Z. Zhuang, H. Liang, L. Peng, H. Wang, and J. Sun. 2021. Self-validation: Early stopping for single-instance deep generative priors. arXiv Preprint arXiv: 2110.12271.
   Google Scholar
• Mahsereci, M., L. Balles, C. Lassner, and P. Hennig. 2017. Early stopping without a validation set. arXiv Preprint arXiv: 1703.09580.
   Google Scholar
• Narkhede, M. V., P. P. Bartakke, and M. S. Sutaone. 2022. A review on weight initialization strategies for neural networks. Artificial Intelligence Review 55 (1):291–322. doi:10.1007/s10462-021-10033-z
   Web of Science ®Google Scholar
• Park, E., J. Ahn, and S. Yoo. 2017. Weighted-entropy-based quantization for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 5456–64.
   Google Scholar
• Pinto, T., H. Morais, and J. M. Corchado. 2019. Adaptive entropy-based learning with dynamic artificial neural network. Neurocomputing 338:432–40. doi:10.1016/j.neucom.2018.09.092
   Web of Science ®Google Scholar
• Podoleanu, A. G. 2012. Optical coherence tomography. Journal of Microscopy 247 (3):209–19. doi:10.1111/j.1365-2818.2012.03619.x
   PubMed Web of Science ®Google Scholar
• Powers, D. M. 2020. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv Preprint arXiv: 2010.16061.
   Google Scholar
• Raskutti, G., M. J. Wainwright, and B. Yu. 2014. Early stopping and non-parametric regression: An optimal data-dependent stopping rule. The Journal of Machine Learning Research 15 (1):335–66.
   Google Scholar
• Rawat, W., and Z. Wang. 2017. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation 29 (9):2352–449. doi:10.1162/neco_a_00990
   PubMed Web of Science ®Google Scholar
• Shannon, C. E. 1948. A mathematical theory of communication. The Bell System Technical Journal 27 (3):379–423. doi:10.1002/j.1538-7305.1948.tb01338.x
   Google Scholar
• Shapiro, S. S., and M. B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52 (3/4):591–611. doi:10.1093/biomet/52.3-4.591
   Web of Science ®Google Scholar
• Silva, L. M., J. M. de Sá, and L. A. Alexandre. 2005. Neural network classification using Shannon’s entropy. In Esann, 217–222. Citeseer.
   Google Scholar
• Simonyan, K., and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv Preprint arXiv: 14091556.
   Google Scholar
• Sinha, A., M. Sarkar, A. Mukherjee, and B. Krishnamurthy. 2017. Introspection: Accelerating neural network training by learning weight evolution. arXiv Preprint arXiv: 1704.04959.
   Google Scholar
• Srinivasan, V., C. Eswaran, and N. Sriraam. 2007. Approximate entropy-based epileptic EEG detection using artificial neural networks. IEEE Transactions on Information Technology in Biomedicine 11 (3):288–95. doi:10.1109/TITB.2006.884369
   PubMedGoogle Scholar
• Steinke, T., and L. Zakynthinou. 2020. Reasoning about generalization via conditional mutual information. In Conference on Learning Theory, Graz, Austria, 3437–52. PMLR.
   Google Scholar
• Sunija, A. P., S. Kar, S. Gayathri, V. P. Gopi, and P. Palanisamy. 2021. Octnet: A lightweight cnn for retinal disease classification from optical coherence tomography images. Computer Methods and Programs in Biomedicine 200:105877. doi:10.1016/j.cmpb.2020.105877
   PubMed Web of Science ®Google Scholar
• Vardasbi, A., M. de Rijke, and M. Dehghani. 2022. Intersection of Parallels as an Early Stopping Criterion. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta,GA, USA, 1965–74.
   Google Scholar
• Xu, Y., P. Cao, Y. Kong, and Y. Wang. 2019. L_dmi: A novel information-theoretic loss function for training deep nets robust to label noise. Advances in Neural Information Processing Systems 32.
   Google Scholar
• Xu, A., and M. Raginsky. 2017. Information-theoretic analysis of generalization capability of learning algorithms. Advances in Neural Information Processing Systems 30.
   Google Scholar
• Yilmaz, A., and R. Poli. 2022. Successfully and efficiently training deep multi-layer perceptrons with logistic activation function simply requires initializing the weights with an appropriate negative mean. Neural Networks 153:87–103. doi:10.1016/j.neunet.2022.05.030
   PubMed Web of Science ®Google Scholar