Microstructure reconstruction using diffusion-based generative models

References

• R. Bostanabad, Y. Zhang, X. Li, T. Kearney, L.C. Brinson, D.W. Apley, W.K. Liu, and W. Chen, Computational microstructure characterization and reconstruction: Review of the state-of-the-art techniques, Prog. Mater. Sci., vol. 95, pp. 1–41, 2018. DOI: 10.1016/j.pmatsci.2018.01.005.
   Web of Science ®Google Scholar
• S. Bargmann, B. Klusemann, J. Markmann, J.E. Schnabel, K. Schneider, C. Soyarslan, and J. Wilmers, Generation of 3D representative volume elements for heterogeneous materials: A review, Prog. Mater. Sci., vol. 96, pp. 322–384, 2018. DOI: 10.1016/j.pmatsci.2018.02.003.
   Web of Science ®Google Scholar
• P. Seibert, A. Raßloff, K. Kalina, M. Ambati, and M. Kästner, Microstructure characterization and reconstruction in Python: MCRpy, Integr. Mater. Manuf. Innov., vol. 11, no. 3, pp. 450–466, 2022. DOI: 10.1007/s40192-022-00273-4.
   Web of Science ®Google Scholar
• P. Seibert, A. Raßloff, M. Ambati, and M. Kästner, Descriptor-based reconstruction of three-dimensional microstructures through gradient-based optimization, Acta Mater., vol. 227, pp. 117667, 2022. DOI: 10.1016/j.actamat.2022.117667.
   Web of Science ®Google Scholar
• X. Li, Z. Yang, L.C. Brinson, A. Choudhary, A. Agrawal, and W. Chen, A deep adversarial learning methodology for designing microstructural material systems, International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, 2018. pp. V02BT03A008. DOI: 10.1115/DETC2018-85633.
   Google Scholar
• M. Sahimi, and P. Tahmasebi, Reconstruction, optimization, and design of heterogeneous materials and media: Basic principles, computational algorithms, and applications, Phys. Rep., vol. 939, pp. 1–82, 2021. DOI: 10.1016/j.physrep.2021.09.003.
   Google Scholar
• B.-S. Lee, and W.-R. Yu, Electrospun carbon nanofibers as a functional composite platform: A review of highly tunable microstructures and morphologies for versatile applications, Funct. Compos. Struct., vol. 2, no. 1, pp. 012001, 2020. DOI: 10.1088/2631-6331/ab7a8c.
   Google Scholar
• D.-W. Kim, S.-M. Park, and J.H. Lim, Prediction of the transverse elastic modulus of the unidirectional composites by an artificial neural network with fiber positions and volume fraction, Funct. Compos. Struct., vol. 3, no. 2, pp. 025003, 2021. DOI: 10.1088/2631-6331/abf8f8.
   Google Scholar
• D. Singh, R. Kumar, and R. Vaish, Finite element-based homogenization model to determine effective properties of 0–3 and 1–3 electrostrictive composite, Funct. Compos. Struct., vol. 4, no. 4, pp. 045007, 2022. DOI: 10.1088/2631-6331/aca753.
   Google Scholar
• H.J. Lim, H. Choi, S.-J. Yoon, S.W. Lim, C.-H. Choi, and G.J. Yun, Micro-CT image-based reconstruction algorithm for multiscale modeling of Sheet Molding Compound (SMC) composites with experimental validation, Compos. Mater. Engin., vol. 3, pp. 221–239, 2021.
   Google Scholar
• A. Cherouat, and H. Borouchaki, Advanced numerical tool for composite woven fabric preforming, Adv. Aircraft Spacecraft Sci., vol. 2, no. 1, pp. 1–16, 2015. DOI: 10.12989/aas.2015.2.1.001.
   Google Scholar
• H. Xu, D.A. Dikin, C. Burkhart, and W. Chen, Descriptor-based methodology for statistical characterization and 3D reconstruction of microstructural materials, Comput. Mater. Sci., vol. 85, pp. 206–216, 2014. DOI: 10.1016/j.commatsci.2013.12.046.
   Web of Science ®Google Scholar
• Y. Jiao, F. Stillinger, and S. Torquato, Modeling heterogeneous materials via two-point correlation functions: Basic principles, Phys. Rev. E Stat. Nonlin. Soft Matter. Phys., vol. 76, no. 3 Pt 1, pp. 031110, 2007. DOI: 10.1103/PhysRevE.76.031110.
   PubMed Web of Science ®Google Scholar
• H. Kumar, C. Briant, and W. Curtin, Using microstructure reconstruction to model mechanical behavior in complex microstructures, Mech. Mater., vol. 38, no. 8–10, pp. 818–832, 2006. DOI: 10.1016/j.mechmat.2005.06.030.
   Web of Science ®Google Scholar
• Z. Jiang, W. Chen, and C. Burkhart, Efficient 3D porous microstructure reconstruction via Gaussian random field and hybrid optimization, J Microsc., vol. 252, no. 2, pp. 135–148, 2013. DOI: 10.1111/jmi.12077.
   PubMed Web of Science ®Google Scholar
• A. Senthilnathan, P. Acar, and M. De Graef, Markov Random Field based microstructure reconstruction using the principal image moments, Mater. Charact., vol. 178, pp. 111281, 2021. DOI: 10.1016/j.matchar.2021.111281.
   Web of Science ®Google Scholar
• R. Bostanabad, A.T. Bui, W. Xie, D.W. Apley, and W. Chen, Stochastic microstructure characterization and reconstruction via supervised learning, Acta Mater., vol. 103, pp. 89–102, 2016. DOI: 10.1016/j.actamat.2015.09.044.
   Web of Science ®Google Scholar
• S. Yu, Y. Zhang, C. Wang, W.-K. Lee, B. Dong, T.W. Odom, C. Sun, and W. Chen, Characterization and design of functional quasi-random nanostructured materials using spectral density function, J. Mech. Des., vol. 139, no. 7, pp. 071401, 2017. DOI: 10.1115/1.4036582.
   Web of Science ®Google Scholar
• A. Iyer, R. Dulal, Y. Zhang, U.F. Ghumman, T. Chien, G. Balasubramanian, and W. Chen, Designing anisotropic microstructures with spectral density function, Comput. Mater. Sci., vol. 179, pp. 109559, 2020. DOI: 10.1016/j.commatsci.2020.109559.
   Web of Science ®Google Scholar
• S. Torquato, Random Heterogeneous Materials, Interdisciplinary Applied Mathematics. Springer, New York, 2013.
   Google Scholar
• C. Yeong, and S. Torquato, Reconstructing random media, Phys. Rev. E., vol. 57, no. 1, pp. 495–506, 1998. DOI: 10.1103/PhysRevE.57.495.
   Web of Science ®Google Scholar
• S. Torquato, Statistical description of microstructures, Annu. Rev. Mater. Res., vol. 32, no. 1, pp. 77–111, 2002. DOI: 10.1146/annurev.matsci.32.110101.155324.
   Google Scholar
• Y. Liu, M.S. Greene, W. Chen, D.A. Dikin, and W.K. Liu, Computational microstructure characterization and reconstruction for stochastic multiscale material design, Comput-Aided. Des., vol. 45, no. 1, pp. 65–76, 2013. DOI: 10.1016/j.cad.2012.03.007.
   Web of Science ®Google Scholar
• P. Seibert, M. Ambati, A. Raßloff, and M. Kästner, Reconstructing random heterogeneous media through differentiable optimization, Comput. Mater. Sci., vol. 196, pp. 110455, 2021. DOI: 10.1016/j.commatsci.2021.110455.
   Web of Science ®Google Scholar
• A. Cecen, B. Yucel, and S.R. Kalidindi, A generalized and modular framework for digital generation of composite microstructures, J. Compos. Sci., vol. 5, no. 8, pp. 211, 2021. DOI: 10.3390/jcs5080211.
   Google Scholar
• X. Li, Y. Zhang, H. Zhao, C. Burkhart, L.C. Brinson, and W. Chen, A transfer learning approach for microstructure reconstruction and structure-property predictions, Sci Rep., vol. 8, no. 1, pp. 13, 2018. DOI: 10.1038/s41598-018-31571-7.
   PubMedGoogle Scholar
• Y. Pu, Z. Gan, R. Henao, X. Yuan, C. Li, A. Stevens, and L. Carin, Variational autoencoder for deep learning of images, labels and captions. In: Advances in Neural Information Processing Systems, 30th Conference on Neural information Processing Systems (NIPS 2016), Barcelona, Spain. vol. 29, 2016.
   Google Scholar
• T. Guo, D.J. Lohan, R. Cang, M.Y. Ren, and J.T. Allison, An indirect design representation for topology optimization using variational autoencoder and style transfer, 2018 AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 2018. pp. 0804. DOI: 10.2514/6.2018-0804.
   Google Scholar
• K.M. Guan, T.I. Anderson, P. Creux, and A.R. Kovscek, Reconstructing porous media using generative flow networks, Comput. Geosci., vol. 156, pp. 104905, 2021. DOI: 10.1016/j.cageo.2021.104905.
   Web of Science ®Google Scholar
• L. Dinh, D. Krueger, and Y. Bengio, Nice: Non-linear independent components estimation, arXiv Preprint arXiv., vol. 00796, 2014.
   Google Scholar
• D.P. Kingma, and P. Dhariwal, Glow: Generative flow with invertible 1x1 convolutions. In: Advances in Neural Information Processing Systems, 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada, vol. 31, 2018.
   Google Scholar
• S. Kench, and S.J. Cooper, Generating 3D structures from a 2D slice with GAN-based dimensionality expansion, arXiv Preprint arXiv., pp. 07708, 2021.
   Google Scholar
• A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A.A. Bharath, Generative adversarial networks: An overview, IEEE Signal Process. Mag., vol. 35, no. 1, pp. 53–65, 2018. DOI: 10.1109/MSP.2017.2765202.
   Web of Science ®Google Scholar
• I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative adversarial networks, Commun. ACM., vol. 63, no. 11, pp. 139–144, 2020. DOI: 10.1145/3422622.
   Web of Science ®Google Scholar
• I. Ferreira, L. Ochoa, and A. Koeshidayatullah, On the generation of realistic synthetic petrographic datasets using a style-based GAN, Sci Rep., vol. 12, no. 1, pp. 14, 2022. DOI: 10.1038/s41598-022-16034-4.
   PubMed Web of Science ®Google Scholar
• A. Henkes, and H. Wessels, Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics, arXiv Preprint arXiv., pp. 01693, 2022.
   Google Scholar
• A. Sauer, K. Schwarz, and A. Geiger, Stylegan-xl: Scaling stylegan to large diverse datasets, ACM SIGGRAPH 2022 Conference Proceedings, 2022. pp. 1–10. DOI: 10.1145/3528233.3530738.
   Google Scholar
• T. Karras, S. Laine, and T. Aila, A style-based generator architecture for generative adversarial networks, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. pp. 4401–4410.
   Google Scholar
• J. Ho, C. Saharia, W. Chan, D.J. Fleet, M. Norouzi, and T. Salimans, Cascaded diffusion models for high fidelity image generation, J. Mach. Learn. Res., Microtome Publishing, Brookline, vol. 23, pp. 47:41–47:33, 2022.
   Web of Science ®Google Scholar
• P. Dhariwal, and A. Nichol, Diffusion models beat gans on image synthesis, Adv. Neural Inform. Process. Syst., 35th Conference on Neural Information Processing Systems (NeurIPS 2021), vol. 34, pp. 8780–8794, 2021.
   Google Scholar
• A.Q. Nichol, and P. Dhariwal, Improved denoising diffusion probabilistic models, International Conference on Machine Learning, PMLR, Proceedings of the 38th International Conference on Machine Learning, 2021, pp. 8162–8171.
   Google Scholar
• M. Armandpour, A. Sadeghian, C. Li, and M. Zhou, Partition-guided gans, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. pp. 5099–5109.
   Google Scholar
• S. Li, Y. Du, J.B. Tenenbaum, A. Torralba, and I. Mordatch, Composing ensembles of pre-trained models via iterative consensus, arXiv Preprint arXiv., pp. 11522, 2022.
   Google Scholar
• S. Lala, M. Shady, A. Belyaeva, and M. Liu, Evaluation of mode collapse in generative adversarial networks. In: High Performance Extreme Computing, 2018.
   Google Scholar
• A. Brock, J. Donahue, and K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, arXiv Preprint arXiv., pp. 11096, 2018.
   Google Scholar
• T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, Spectral normalization for generative adversarial networks, arXiv Preprint arXiv., pp. 05957, 2018.
   Google Scholar
• L. Wang, W. Chen, W. Yang, F. Bi, and F.R. Yu, A state-of-the-art review on image synthesis with generative adversarial networks, IEEE Access., vol. 8, pp. 63514–63537, 2020. DOI: 10.1109/ACCESS.2020.2982224.
   Google Scholar
• H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, Self-attention generative adversarial networks, International Conference on Machine Learning, PMLR, 2019, pp. 7354–7363.
   Google Scholar
• J. Song, C. Meng, and S. Ermon, Denoising diffusion implicit models, arXiv Preprint arXiv., pp. 02502, 2020.
   Google Scholar
• J. Ho, A. Jain, and P. Abbeel, Denoising diffusion probabilistic models, Adv. Neural Inform. Process. Syst., vol. 33, pp. 6840–6851, 2020.
   Google Scholar
• L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, Diffusion models: A comprehensive survey of methods and applications, arXiv Preprint arXiv., pp. 00796, 2022.
   Google Scholar
• J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, International Conference on Machine Learning, PMLR, Proceedings of the 32nd International Conference on Machine Learning, Lille, France.2015. pp. 2256–2265.
   Google Scholar
• H. Cao, C. Tan, Z. Gao, G. Chen, P.-A. Heng, and S.Z. Li, A survey on generative diffusion model, arXiv Preprint arXiv., pp. 02646, 2022.
   Google Scholar
• Y. Song, and S. Ermon, Generative modeling by estimating gradients of the data distribution. In: Advances in Neural Information Processing Systems, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. vol. 32, 2019.
   Google Scholar
• S. Mohamed, and B. Lakshminarayanan, Learning in implicit generative models, arXiv Preprint arXiv., pp. 03483, 2016.
   Google Scholar
• M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30, 2017.
   Google Scholar
• T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila, Improved Precision and Recall Metric for Assessing Generative Models, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. vol. 32, 2019.
   Google Scholar
• K.A. Hart, and J.J. Rimoli, MicroStructPy: A statistical microstructure mesh generator in Python, SoftwareX., vol. 12, pp. 100595, 2020. DOI: 10.1016/j.softx.2020.100595.
   Web of Science ®Google Scholar
• O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2015. pp. 234–241.
   Google Scholar
• T. Salimans, A. Karpathy, X. Chen, and D.P. Kingma, Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications, arXiv Preprint arXiv., pp. 05517, 2017.
   Google Scholar
• C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the inception architecture for computer vision, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. pp. 2818–2826.
   Google Scholar
• A. Borji, Pros and cons of GAN evaluation measures: New developments, Computer Vision Image Understanding., vol. 215, pp. 103329, 2022. DOI: 10.1016/j.cviu.2021.103329.
   Web of Science ®Google Scholar
• M.F. Naeem, S.J. Oh, Y. Uh, Y. Choi, and J. Yoo, Reliable fidelity and diversity metrics for generative models, International Conference on Machine Learning, PMLR, 2020. pp. 7176–7185.
   Google Scholar
• B. Lu, and S. Torquato, Lineal-path function for random heterogeneous materials, Phys Rev A., vol. 45, no. 2, pp. 922–929, 1992. DOI: 10.1103/physreva.45.922.
   PubMed Web of Science ®Google Scholar
• A. Cecen, T. Fast, and S.R. Kalidindi, Versatile algorithms for the computation of 2-point spatial correlations in quantifying material structure, Integr Mater Manuf Innov., vol. 5, no. 1, pp. 1–15, 2016. DOI: 10.1186/s40192-015-0044-x.
   Google Scholar
• R. Bostanabad, W. Chen, and D. Apley, Characterization and reconstruction of 3D stochastic microstructures via supervised learning, J Microsc., vol. 264, no. 3, pp. 282–297, 2016. DOI: 10.1111/jmi.12441.
   PubMed Web of Science ®Google Scholar
• C. Meehan, K. Chaudhuri, and S. Dasgupta, A non-parametric test to detect data-copying in generative models, International Conference on Artificial Intelligence and Statistics, 2020.
   Google Scholar
• D.P. Kingma, and J. Ba, Adam: A method for stochastic optimization, arXiv Preprint arXiv., vol. 1412, pp. 6980, 2014.
   Google Scholar
• A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. vol. 32, 2019.
   Google Scholar
• NJSm Hansen, Hall–Petch relation and boundary strengthening, Scr. Mater., vol. 51, no. 8, pp. 801–806, 2004. DOI: 10.1016/j.scriptamat.2004.06.002.
   Web of Science ®Google Scholar
• F. Najafkhani, S. Kheiri, B. Pourbahari, and H. Mirzadeh, Recent advances in the kinetics of normal/abnormal grain growth: a review, Archiv. Civil Mech. Engin., vol. 21, pp. 1–20, 2021.
   Web of Science ®Google Scholar
• A. Borbély, F. Csikor, S. Zabler, P. Cloetens, and H. Biermann, Three-dimensional characterization of the microstructure of a metal–matrix composite by holotomography, Mater. Sci. Engin.: A., vol. 367, no. 1–2, pp. 40–50, 2004. DOI: 10.1016/j.msea.2003.09.068.
   Web of Science ®Google Scholar
• Y. Feng, F. Thiemann, and M. Sester, Learning cartographic building generalization with deep convolutional neural networks, IJGI., vol. 8, no. 6, pp. 258, 2019. DOI: 10.3390/ijgi8060258.
   Google Scholar
• H. Emami, M. Dong, S. Nejad-Davarani, and C. Glide-Hurst, SA-GAN: Structure-aware generative adversarial network for shape-preserving synthetic CT generation, arXiv Preprint arXiv., pp. 07044, 2021.
   Google Scholar
• Y.-H. Su, W. Jiang, D. Chitrakar, K. Huang, H. Peng, and B. Hannaford, Local style preservation in improved gan-driven synthetic image generation for endoscopic tool segmentation, Sensors., vol. 21, no. 15, pp. 5163, 2021. DOI: 10.3390/s21155163.
   PubMed Web of Science ®Google Scholar
• A. Gokaslan, V. Ramanujan, D. Ritchie, K.I. Kim, and J. Tompkin, Improving shape deformation in unsupervised image-to-image translation, Proceedings of the European Conference on Computer Vision (ECCV), 2018. pp. 649–665.
   Google Scholar
• M. Kwon, J. Jeong, and Y. Uh, Diffusion models already have a semantic latent space, arXiv Preprint arXiv., pp. 10960, 2022.
   Google Scholar
• R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, High-resolution image synthesis with latent diffusion models, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. pp. 10684–10695.
   Google Scholar
• M.D. Rintoul, and S. Torquato, Reconstruction of the structure of dispersions, J Colloid Interface Sci., vol. 186, no. 2, pp. 467–476, 1997. DOI: 10.1006/jcis.1996.4675.
   PubMed Web of Science ®Google Scholar
• M. Tarantino, O. Zerhouni, and K. Danas, Random 3D-printed isotropic composites with high volume fraction of pore-like polydisperse inclusions and near-optimal elastic stiffness, Acta Mater., vol. 175, pp. 331–340, 2019. DOI: 10.1016/j.actamat.2019.06.020.
   Web of Science ®Google Scholar
• N. Lubbers, T. Lookman, and K. Barros, Inferring low-dimensional microstructure representations using convolutional neural networks, Phys. Rev. E., vol. 96, no. 5-1, pp. 052111, 2017. DOI: 10.1103/PhysRevE.96.052111.
   PubMed Web of Science ®Google Scholar
• S.G. Nash, Newton-type minimization via the Lanczos method, SIAM J. Numer. Anal., vol. 21, no. 4, pp. 770–788, 1984. DOI: 10.1137/0721052.
   Web of Science ®Google Scholar
• R.H. Byrd, S.L. Hansen, J. Nocedal, and Y. Singer, A stochastic quasi-Newton method for large-scale optimization, SIAM J. Optim., vol. 26, no. 2, pp. 1008–1031, 2016. DOI: 10.1137/140954362.
   Web of Science ®Google Scholar