Inverse design of convolutional neural networks via nanophotonic kernels

References

• Ching T, Himmelstein DS, Beaulieu-Jones BK, et al. Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface. 2018;15(141):20170387. doi: 10.1098/rsif.2017.0387
   PubMed Web of Science ®Google Scholar
• Guest D, Cranmer K, Whiteson D. Deep learning and its application to LHC physics. Ann Rev Nucl Particle Sci. 2018;68:161–181. doi: 10.1146/annurev-nucl-101917-021019
   Web of Science ®Google Scholar
• Purwins H, Li B, Virtanen T, et al. Deep learning for audio signal processing. IEEE J Sel Top Sig Proc. 2019;13(2):206–219. doi: 10.1109/JSTSP.2019.2908700
   Web of Science ®Google Scholar
• Meng J, Miscuglio M, George JK, et al. Electronic bottleneck suppression in next-generation networks with integrated photonic digital-to-analog converters. Adv Photonics Re. 2021;2(2):2000033. doi: 10.1002/adpr.202000033
   Google Scholar
• Huang C, Sorger VJ, Miscuglio M, et al. Prospects and applications of photonic neural networks. Adv Phys X. 2022;7(1):1981155. doi: 10.1080/23746149.2021.1981155
   Google Scholar
• Shen Y, Harris NC, Skirlo S, et al. Deep learning with coherent nanophotonic circuits. Nat Photonics. 2017;11(7):441–446. doi: 10.1038/nphoton.2017.93
   Web of Science ®Google Scholar
• Xu S, Wang J, Zou W. Optical convolutional neural network with WDM-based optical patching and microring weighting banks. IEEE Photonics Technol Lett. 2020;33(2):89–92. doi: 10.1109/LPT.2020.3045478
   Web of Science ®Google Scholar
• Feldmann J, Youngblood N, Karpov M, et al. Parallel convolutional processing using an integrated photonic tensor core. Nature. 2021;589(7840):52–58. doi: 10.1038/s41586-020-03070-1
   PubMed Web of Science ®Google Scholar
• Lin X, Rivenson Y, Yardimci NT, et al. All-optical machine learning using diffractive deep neural networks. Science. 2018;361(6406):1004–1008. doi: 10.1126/science.aat8084
   PubMed Web of Science ®Google Scholar
• Hamerly R, Bernstein L, Sludds A, et al. Large-scale optical neural networks based on photoelectric multiplication. Phys Rev X. 2019;9(2):021032. doi: 10.1103/PhysRevX.9.021032
   Web of Science ®Google Scholar
• Rahman MSS, Li J, Mengu D, et al. Ensemble learning of diffractive optical networks. Light Sci Appl. 2021;10(1):1–13. doi: 10.1038/s41377-020-00435-z
   PubMed Web of Science ®Google Scholar
• Xu X, Tan M, Corcoran B, et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature. 2021; 589(7840): 44–51. doi: 10.1038/s41586-020-03063-0
   PubMed Web of Science ®Google Scholar
• Zhang H, Gu M, Jiang XD, et al. An optical neural chip for implementing complex-valued neural network. Nat Commun. 2021;12(1):457. doi: 10.1038/s41467-020-20719-7
   PubMed Web of Science ®Google Scholar
• Ashtiani F, Geers AJ, Aflatouni F. An on-chip photonic deep neural network for image classification. Nature. 2022; 606(7914): 501–506. doi: 10.1038/s41586-022-04714-0
   PubMed Web of Science ®Google Scholar
• Nakajima M, Tanaka K, Hashimoto T. Scalable reservoir computing on coherent linear photonic processor. Commun Physics. 2021;4(1):20. doi: 10.1038/s42005-021-00519-1
   Web of Science ®Google Scholar
• Molesky S, Lin Z, Piggott AY, et al. Inverse design in nanophotonics. Nat Photonics. 2018;12(11):659–670. doi: 10.1038/s41566-018-0246-9
   Web of Science ®Google Scholar
• Liu Z, Zhu D, Raju L, et al. Tackling photonic inverse design with machine learning. Adv Sci. 2021;8(5):2002923. doi: 10.1002/advs.202002923
   Web of Science ®Google Scholar
• Vercruysse D, Sapra NV, Su L, et al. Dispersion engineering with photonic inverse design. IEEE J Sel Top Quantum Electron. 2020;26(2):1–6. doi: 10.1109/JSTQE.2019.2950803
   Web of Science ®Google Scholar
• Qu Y, Zhu H, Shen Y, et al. Inverse design of an integrated-nanophotonics optical neural network. Sci Bull. 2020;65(14):1177–1183. doi: 10.1016/j.scib.2020.03.042
   PubMed Web of Science ®Google Scholar
• Mao S, Cheng L, Khan FN, et al. Inverse design of high-dimensional nanostructured 2× 2 optical processors based on deep convolutional neural networks. J Lightwave Technol. 2022;40(9):2926–2932. doi: 10.1109/JLT.2022.3147018
   Web of Science ®Google Scholar
• Oubre C, Nordlander P. Optical properties of metallodielectric nanostructures calculated using the finite difference time domain method. J Phys Chem B. 2004;108(46):17740–17747. doi: 10.1021/jp0473164
   Web of Science ®Google Scholar
• Hughes TW, Minkov M, Williamson IAD, et al. Adjoint method and inverse design for nonlinear nanophotonic devices. ACS Photonics. 2018;5(12):4781–4787. doi: 10.1021/acsphotonics8b01522
   Web of Science ®Google Scholar
• Zhu C, Byrd RH, Lu P, et al. Algorithm 778: L-BFGS-B: fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw. 1997;23(4):550–560. doi: 10.1145/279232.279236
   Web of Science ®Google Scholar
• Andrews H, Patterson C. Singular value decompositions and digital image processing. IEEE Trans Acoustics Speech Sig Proc. 1976;24(1):26–53. doi: 10.1109/TASSP.1976.1162766
   Google Scholar
• Boureau YL, Ponce J, LeCun Y. A theoretical analysis of feature pooling in visual recognition. In Proceedings of the 27th international conference on machine learning (ICML-10). 2010; 111–118.
   Google Scholar
• Sainath TN, Vinyals O, Senior A, et al. Convolutional, long short-term memory, fully connected deep neural networks. In 2015 IEEE international conference on acoustics, speech and signal processing. 2015; 4580–4584.
   Google Scholar
• Deng L. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process Mag. 2012;29(6):141–142. doi: 10.1109/MSP.2012.2211477
   Google Scholar
• Wang Y, Tang H, Xie Y, et al. An in-memory computing architecture based on two-dimensional semiconductors for multiply-accumulate operations. Nat Commun. 2021;12(1):3347. doi: 10.1038/s41467-021-23719-3
   PubMed Web of Science ®Google Scholar
• Wu C, Yu H, Lee S, et al. Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network. Nat Commun. 2021;12(1):96–103. doi: 10.1038/s41467-020-20365-z
   PubMed Web of Science ®Google Scholar