PROGRAMMING AGENT BEHAVIOR BY LEARNING IN SIMULATION MODELS

Figures & data

FIGURE 1 Scenario and behavior example—five agents.

FIGURE 2 Agent perception sectors.

TABLE 1 Mean Number of Collisions Per Run - Rows Represent the Number of Agents

Agent no.	Basic setup	Improved parameters	Fixed positions
1	0.61 ± 0.98	0.22 ± 0.58	0.006 ± 0.10
2	1.21 ± 2.30	0.73 ± 1.71	0.43 ± 0.90
5	2.66 ± 2.21	1.95 ± 2.11	1.23 ± 1.37
10	4.45 ± 2.02	3.65 ± 1.99	3.21 ± 1.81

FIGURE 3 Development of the number of collisions for an exemplary run with five agents.

FIGURE 4 Exemplary trajectories during exploit trials, for two agents.

FIGURE 5 Q-learning value distribution for an exemplary agent from a simulation with one agent.

FIGURE 6 Q-learning value distribution for an exemplary agent from a simulation with five agents.

FIGURE 7 Situation-action pairs experience distribution for an exemplary agent from a simulation with five agents.

FIGURE 8 Situation-action pairs distribution over the actions for an exemplary agent from a simulation with five agents.

TABLE 2 Classification Accuracy with C4.5 Decision Tree Algorithm, all Models

	10 fold cross validation	Training set	Other agent
	Basic model
1 agent	0.48	0.78	0.52
2 agents	0.50	0.73	0.47
5 agents	0.48	0.75	0.52
10 agents	0.53	0.76	0.48
	Improved parameters
1 agent	0.85	0.88	0.79
2 agents	0.82	0.85	0.82
5 agents	0.78	0.86	0.74
10 agents	0.75	0.84	0.83

FIGURE 9 Confusion matrix for the decision tree in the simulation with five agents, basic model. Rows represent the expected class (action) and columns represent the classification determined by the decision tree.

FIGURE 10 Decision tree generated for an exemplary agent of the 5-agents case, basic model.

FIGURE 11 Decision tree generated for an exemplary agent of the 5-agents case, improved parameters model.

FIGURE 12 Decision tree generated for an exemplary agent of the 5-agents case, fixed initial positions model.