Deep reinforcement learning and 3D physical environments applied to crowd evacuation in congested scenarios

Figures & data

Figure 1. Framework for crowd evacuation modeling and simulation by using deep reinforcement learning and 3D physical environments.

Figure 2. Modeling of crowd evacuation scenarios.

Figure 3. Modeling of agents’ perceptron. (a) Vertical field of the agents’ vision; (b) horizontal field of the agents’ vision; (c) schematic diagram of the vision-like ray perceptron. Only perceptual rays in the horizontal plane are shown.

Figure 4. Training-learning mode of PPO.

Figure 5. Double branch feature extraction combined actor and critic network.

Table 1. Configuration of parameters.

Parameters	Value
Discount factor ($\gamma$)	$0.99$
GAE parameter ($\lambda$)	$0.95$
Polynomial decay parameters (${\epsilon }_{high}$, ${\epsilon }_{low}$, $p$)	$0.2,0.1,1$
Loss function parameters ($c_1$, $c_2$)	$0.5,0.005$
Max_step	$1.5\times {10}^5$
Buffer_size	$10240$
Learning_rate	$1\times {10}^{-3}$
Batch_size	$512$
Num_epoch	3

Figure 6. Simulation in the scenario with one exit. (a-d) stills of the simulation results. The temporal sequence of these stills is labeled alphabetically.

Figure 7. Density maps of crowd evacuations. (a) Density map generated by the real crowd experiment; (b) density map generated by our simulation.

Figure 8. Fundamental diagrams of crowd evacuations. (a) fundamental diagram generated by the real crowd experiment; (b) fundamental diagram generated by our simulation.

Figure 9. Impact of various factors on crowd evacuation in S1Es. (a) width of exit; (b) number of agents; (c) mass of agents; (d) height of agents. (a-b) represent the relationship between the number of evacuated agents and the evacuation time.

Figure 10. Relationship between evacuation efficiency and exit width.

Figure 11. Simulations in the scenarios with two exits and a uniform distribution of agents. (a–c) Stills of the simulation results in different S2Es. (a) S2E:L0.5-R0.5; (b) S2E:L0.8-R0.5; (c) S2E:L0.5-R0.8.

Figure 12. Density maps of simulation results in different S2Es. (a) S2E:L0.5-R0.5; (b) S2E:L0.8-R0.5; (c) S2E:L0.5-R0.8.

Figure 13. Relationship between the initial position and evacuation exit in different S2Es. (a) S2E:L0.5-R0.5; (b) S2E:L0.8-R0.5; (c) S2E:L0.5-R0.8. (a–c) The black dotted lines indicate that the distances to the left and right exits are equal.

Table 2. Evacuation statistics of two exits in S2Es.

Scenario	Left exit (%)	Right exit (%)	Difference (%)	Variation (%)
S2E:L0.5-R0.5	49.9	50.1	−0.2	–
S2E:L0.8-R0.5	59.8	40.2	19.6	19.8
S2E:L0.5-R0.8	40.4	59.6	−19.2	−19.0

Figure 14. Simulations in the scenarios with two exits and a nonuniform distribution of agents. (a-c) Stills of the simulation results in different NS2Es. (a) NS2E:L0.5-R0.5; (b) NS2E:L0.8-R0.5; (c) NS2E:L0.5-R0.8.

Figure 15. Density maps of simulation results in NS2Es. (a) NS2E:L0.5-R0.5; (b) NS2E:L0.8-R0.5; (c) NS2E:L0.5-R0.8.

Table 3. Evacuation statistics of two exits in NS2Es.

Scenario	Left exit (%)	Right exit (%)	Difference (%)	Variation (%)
NS2E:L0.5-R0.5	42.3	57.7	−15.4	–
NS2E:L0.8-R0.5	44.6	55.4	−10.8	4.6
NS2E:L0.5-R0.8	35.8	64.2	−28.4	−13.0

Figure 16. Impact of agent height on evacuation in the scenarios with two exits. (a) S2E:L0.5-R0.5; (b) NS2E:L0.5-R0.5. (a-b) represent the relationship between the number of evacuated agents and the evacuation time.

Figure 17. Relationship between the initial position and evacuation exit in different NS2Es. (a) NS2E:L0.5-R0.5; (b) NS2E:L0.8-R0.5; (c) NS2E:L0.5-R0.8. (a-c) The black dotted lines indicate that the distances to the left and right exits are equal.

Table 4. Evacuation times in NS2Es.

Scenario	Evacuation time (s)	variation (s)
NS2E:L0.5-R0.5	50.62	–
NS2E:L0.8-R0.5	49.22	−1.40
NS2E:L0.5-R0.8	36.85	−13.77

Data availability statement

The data that support the findings of this study can be obtained from the corresponding author upon reasonable request.