Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

References

• Clancy NT, Jones G, Maier-Hein L, Elson DS, Stoyanov D. 2020. Surgical spectral imaging. Med Image Anal. 63:101699. doi:https://doi.org/10.1016/j.media.2020.101699.
   Google Scholar
• Dijkstra K, van de Loosdrecht J, Schomaker L, Wiering MA. 2019. Hyperspectral demosaicking and crosstalk correction using deep learning. Mach Vis Appl. 30(1):1–21. doi:https://doi.org/10.1007/s00138-018-0965-4.
   Google Scholar
• Ebner M, Nabavi E, Shapey J, Xie Y, Liebmann F, Spirig JM, Hoch A, Farshad M, Saeed SR, Bradford R, et al. 2021. Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation. J Phys D: Appl Phys. 54(29):294003. doi:https://doi.org/10.1088/1361-6463/abfbf6.
   Google Scholar
• Fabelo H, Ortega S, Szolna A, Bulters D, Piñeiro JF, Kabwama S, J-O’Shanahan A, Bulstrode H, Bisshopp S, Kiran BR, et al. 2019. In-vivo hyperspectral human brain image database for brain cancer detection. IEEE Access. 7:39098–39116. doi:https://doi.org/10.1109/ACCESS.2019.2904788.
   Google Scholar
• Geelen B, Tack N, Lambrechts A. 2014. A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic. In: Advanced fabrication technologies for micro/nano optics and photonics VII; vol. 8974. International Society for Optics and Photonics, San Francisco, California, United States. p. 89740L.
   Google Scholar
• Hyttinen J, Fält P, Jäsberg H, Kullaa A, Hauta-Kasari M. 2020. Oral and dental spectral image database—odsi-db. Appl Sci. 10(20):7246. doi:https://doi.org/10.3390/app10207246.
   Google Scholar
• Johnson J, Alahi A, Fei-Fei L. 2016. Perceptual losses for real-time style transfer and super- resolution. In: European Conference on Computer Vision, Amsterdam, The Netherlands. Springer. p. 694–711.
   Google Scholar
• Kaale AJ, Rutabasibwa N, Mchome LL, Lillehei KO, Honce JM, Kahamba J, Ormond DR. 2021. The use of intraoperative neurosurgical ultrasound for surgical navigation in low- and middle-income countries: the initial experience in Tanzania. J Neurosurg JNS. 134(2):630–637. https://thejns.org/view/journals/j-neurosurg/134/2/article-p630.xml.
   Google Scholar
• Kerfoot E, Clough J, Oksuz I, Lee J, King AP, Schnabel JA. 2019. Left-ventricle quantification using residual u-net. In: Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges. Cham: Springer International Publishing. p. 371–380.
   Google Scholar
• Kingma D, Ba J 2014. Adam: a method for stochastic optimization. In: International Conference on Learning Representations, San Diego, California, United States.
   Google Scholar
• Lu G, Fei B. 2014. Medical hyperspectral imaging: a review. J Biomed Opt. 19(1):010901. doi:https://doi.org/10.1117/1.JBO.19.1.010901.
   Google Scholar
• Lugmayr A, Danelljan M, Timofte R 2020. NTIRE 2020 challenge on real-world image super- resolution: methods and results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, Washington, United States. p. 494–495.
   Google Scholar
• Pichette J, Goossens T, Vunckx K, Lambrechts A. 2017. Hyperspectral calibration method for CMOS-based hyperspectral sensors. In: Soskind YG, Olson C, editors. Photonic instrumentation engineering IV; vol. 10110. International Society for Optics and Photonics, San Francisco, California, United States; SPIE; p. 132–144.
   Google Scholar
• Ronneberger O, Fischer P, Brox T. 2015. U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer- Assisted Intervention, Munich, Germany. Springer. p. 234–241.
   Google Scholar
• Shapey J, Xie Y, Nabavi E, Bradford R, Saeed SR, Ourselin S, Vercauteren T. 2019. Intraoperative multispectral and hyperspectral label-free imaging: a systematic review of in vivo clinical studies. J Biophoton. 12(9):e201800455. doi:https://doi.org/10.1002/jbio.201800455.
   Google Scholar
• Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, San Diego, California, United States. p. 1–14.
   Google Scholar
• Tetschke F, Markgraf W, Gransow M, Koch S, Thiele C, Kulcke A, Malberg H. 2016. Hyperspectral imaging for monitoring oxygen saturation levels during normothermic kidney perfusion. J Sens Sens Syst. 5(2):313–318. doi:https://doi.org/10.5194/jsss-5-313-2016.
   Google Scholar
• Valdes PA, Juvekar P, Agar NYR, Gioux S, Golby AJ. 2019. Quantitative wide-field imaging techniques for fluorescence guided neurosurgery. Front Surg. 6:31. https://www.frontiersin.org/article/10.3389/fsurg.2019.00031.
   Google Scholar
• Wagadarikar A, John R, Willett R, Brady D. 2008. Single disperser design for coded aperture snapshot spectral imaging. Appl Opt. 47(10):B44–B51. doi:https://doi.org/10.1364/AO.47.000B44.
   Google Scholar
• Zhang R, Isola P, Efros AA, Shechtman E, Wang O. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, United States. p. 586–595.
   Google Scholar
• Zhuang L, Bioucas-Dias JM. 2018. Hy-demosaicing: hyperspectral blind reconstruction from spectral subsampling. In: IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain. p. 4015–4018.
   Google Scholar