ANN-based methods for solving partial differential equations: a survey

Figures & data

Figure 1. ANN Perceptron 1 (a) and multi-layered perceptron (MLP) 1 (b) architectures.

Figure 2. Illustration of the internal, boundary and initial points of $m=6\times n=7.$

Table 1. The performance of PyDens, NeuroDiffeq and Nangs methods for solving PDE heat equation.

Training data	FDM		Pydens		NeuroDiffeq		Nangs
	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss
25 × 25	2.87	$3.04\times {10}^{-4}$	53	$2.6\times {10}^{-4}$	65	$1.52\times {10}^{-3}$	578	$2.41\times {10}^{-3}$
50 × 50	6.81	$2.67\times {10}^{-5}$	116	$7.59\times {10}^{-6}$	231	$2.97\times {10}^{-5}$	602	$5.49\times {10}^{-4}$
75 × 75	5.54	$2.31\times {10}^{-6}$	203	$8.52\times {10}^{-6}$	480	$2.25\times {10}^{-5}$	726	$4.61\times {10}^{-5}$
100 × 100	5.49	$5.56\times {10}^{-8}$	386	$7.91\times {10}^{-6}$	943	$7.02\times {10}^{-6}$	956	$8.74\times {10}^{-6}$

Figure 3. Analytical solution of heat equation.

Figure 4. Comparison analytical and approximate solutions of PDE heat equation by using Pydens method.

Figure 5. Comparison analytical and approximate solutions of PDE heat equation by using NeuroDiffeq method.

Figure 6. Comparison analytical and approximate solutions of PDE heat equation by using Nangs method.

Figure 7. Comparisons of the loss values and the computational times between the ANN-based methods for solving heat equation at all training points.

Table 2. The loss values of PDE Heat simulation for different number of layers and neurons.

Methods	Layers$\backslash$neurons	32	64	96
Pydens	3	$7.9\times {10}^{-6}$	$9.8\times {10}^{-3}$	$4.9\times {10}^{-5}$
	4	$1.1\times {10}^{-5}$	$9.4\times {10}^{-6}$	$8.1\times {10}^{-6}$
	5	$9.4\times {10}^{-6}$	$9.7\times {10}^{-3}$	$6.6\times {10}^{-6}$
NeuroDiffEq	3	$7.0\times {10}^{-6}$	$1.2\times {10}^{-6}$	$1.2\times {10}^{-6}$
	4	$8.5\times {10}^{-6}$	$3.6\times {10}^{-6}$	$4.9\times {10}^{-6}$
	5	$4.8\times {10}^{-6}$	$6.5\times {10}^{-7}$	$9.2\times {10}^{-7}$
Nangs	3	$8.7\times {10}^{-6}$	$2.8\times {10}^{-6}$	$2.0\times {10}^{-6}$
	4	$9.5\times {10}^{-7}$	$4.6\times {10}^{-7}$	$1.8\times {10}^{-6}$
	5	$2.4\times {10}^{-6}$	$1.4\times {10}^{-6}$	$2.1\times {10}^{-7}$

Table 3. The performance of the three methods for solving PDE wave equation.

Training data	FDM		Pydens		NeuroDiffeq		Nangs
	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss
25 × 25	3.61	$1.01\times {10}^{-3}$	61	$2.82\times {10}^{-3}$	81	$4.33\times {10}^{-2}$	624	$1.84\times {10}^{-1}$
50 × 50	4.39	$2.50\times {10}^{-4}$	150	$2.43\times {10}^{-2}$	284	$1.21\times {10}^{-3}$	707	$7.38\times {10}^{-2}$
75 × 75	5.34	$1.11\times {10}^{-4}$	317	$7.84\times {10}^{-3}$	516	$2.38\times {10}^{-5}$	875	$1.85\times {10}^{-2}$
100 × 100	7.06	$6.21\times {10}^{-5}$	530	$3.42\times {10}^{-2}$	1274	$1.01\times {10}^{-5}$	1094	$1.63\times {10}^{-3}$

Figure 8. Analytical solution of wave equation.

Figure 9. Comparison analytical and approximate solutions of PDE wave equation by using Pydens method.

Figure 10. Comparison analytical and approximate solutions of PDE wave equation by using NeuroDiffeq method.

Figure 11. Comparison analytical and approximate solutions of PDE wave equation by using Nangs method.

Figure 12. Comparisons of the loss values and the computational times between the ANN-based methods for solving wave equation at all training points.

Table 4. Wave simulation results with different number of layers and neurons.

Methods	Layers$\backslash$neurons	32	64	96
Pydens method	3	$3.42\times {10}^{-2}$	$1.77\times {10}^{-2}$	$2.15\times {10}^{-2}$
	4	$2.46\times {10}^{-2}$	$5.9\times {10}^{-3}$	$3.92\times {10}^{-2}$
	5	$1.04\times {10}^{-1}$	$2.6\times {10}^{-2}$	$2.64\times {10}^{-2}$
NeuroDiffEq method	3	$1.0\times {10}^{-5}$	$1.9\times {10}^{-5}$	$5.0\times {10}^{-4}$
	4	$8.5\times {10}^{-6}$	$3.6\times {10}^{-6}$	$2.12\times {10}^{-4}$
	5	$1.9\times {10}^{-3}$	$8.1\times {10}^{-3}$	$2.8\times {10}^{-3}$
Nangs method	3	$1.6\times {10}^{-3}$	$2.1\times {10}^{-4}$	$2.3\times {10}^{-4}$
	4	$5.1\times {10}^{-3}$	$3.2\times {10}^{-4}$	$1.0\times {10}^{-4}$
	5	$1.18\times {10}^{-2}$	$7.0\times {10}^{-5}$	$2.7\times {10}^{-3}$

Figure 13. Analytical solution of Poisson equation.

Table 5. The performance of the three methods for solving Poisson equation.

Training data	FDM		Pydens		NeuroDiffeq		Nangs
	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss	Time (s)	Loss
25 × 25	3.41	$1.14\times {10}^{-6}$	53	$1.69\times {10}^{-4}$	105	$3.42\times {10}^{-7}$	544	$2.92\times {10}^{-3}$
50 × 50	4.34	$3.46\times {10}^{-7}$	126	$2.82\times {10}^{-4}$	391	$2.81\times {10}^{-8}$	653	$4.43\times {10}^{-4}$
75 × 75	11.14	$4.87\times {10}^{-6}$	250	$2.16\times {10}^{-4}$	656	$1.66\times {10}^{-9}$	760	$1.21\times {10}^{-4}$
100 × 100	12.26	$1.30\times {10}^{-4}$	423	$2.23\times {10}^{-4}$	1177	$4.17\times {10}^{-10}$	967	$5.83\times {10}^{-5}$

Figure 14. Comparison analytical and approximate solutions of PDE Poisson by using Pydens method.

Figure 15. Comparison analytical and approximate solutions of PDE Poisson by using NeuroDiffeq method.

Figure 16. Comparison analytical and approximate solutions of PDE Poisson by using Nangs method.

Figure 17. Comparisons of the loss values and the computational times between the ANN-based methods for solving Poisson equation at all training points.

Table 6. Poisson simulation results with different number of layers and neurons.

Methods	Layers$\backslash$neurons	32	64	96
Pydens method	3	$2.2\times {10}^{-4}$	$2.4\times {10}^{-4}$	$1.8\times {10}^{-4}$
	4	$1.9\times {10}^{-4}$	$3.0\times {10}^{-4}$	$3.2\times {10}^{-4}$
	5	$3.1\times {10}^{-4}$	$2.7\times {10}^{-4}$	$1.4\times {10}^{-4}$
NeuroDiffEq method	3	$4.2\times {10}^{-10}$	$2.7\times {10}^{-10}$	$4.4\times {10}^{-10}$
	4	$4.4\times {10}^{-10}$	$3.3\times {10}^{-10}$	$1.3\times {10}^{-9}$
	5	$6.0\times {10}^{-10}$	$5.1\times {10}^{-10}$	$2.3\times {10}^{-9}$
Nangs method	3	$5.8\times {10}^{-5}$	$1.2\times {10}^{-5}$	$1.4\times {10}^{-5}$
	4	$1.8\times {10}^{-5}$	$1.1\times {10}^{-5}$	$6.1\times {10}^{-6}$
	5	$1.3\times {10}^{-5}$	$7.7\times {10}^{-6}$	$5.6\times {10}^{-6}$

Data availability statement

Not applicable.