Map completion from partial observation using the global structure of multiple environmental maps

Figures & data

Figure 1. Overview of the proposed method: map completion network-based SLAM (MCN-SLAM). MCN-SLAM acquires knowledge of the global room structure from the known environments and transfers it to the unknown environment. Global knowledge includes features represented by occupancy grid maps, such as wall layout and room size. The map is complemented by the interaction of global knowledge with partially generated maps using SLAM in the newly visited environment.

Figure 2. The proposed method, map completion network-based SLAM (MCN-SLAM), trains a network to model the global structure extracted from multiple environmental maps using GANs and estimates the unobserved areas based on partial observation. MCN-SLAM extracts the global structure of maps of multiple environments as a prior distribution for map completion. In a newly visited environment, the robot estimates the map using that distribution and observation information. Sensor information and odometry information are used to construct the uncomplete map. The complete map is obtained as a sample from the conditional distribution.

Figure 3. Graphical model representation of the MCN-SLAM generative process where the gray nodes indicate observation variables and the white nodes unobserved variables. The part surrounded by the blue dotted frame is the model representation of the SLAM of one environment. The global knowledge is assumed as a parameter of the prior distribution for maps that integrate local knowledge in various environments.

Table 1. Definition of the variables in the graphical model of MCN-SLAM.

Symbol	Definition
m	Map of the environment
$x_t$	Self-position of the robot
$u_t$	Control observation
$z_t$	Distance observation
α	Parameter of the prior distribution of the map
M	Number of environments

Figure 4. Examples of the dataset: (a) example of the environment maps from the HouseExpo dataset using sufficient observation, (b) example of environment maps from the HouseExpo dataset using insufficient observation of HouseExpo, (c) example of environment maps from the HOME'S dataset using sufficient observation, and (d) example of environment maps from the HOME'S dataset using insufficient observation.

Figure 5. Experiment 1: example of generated maps using the HouseExpo dataset as training data.

Table 2. Experiment 1: experimental results using the HouseExpo dataset as test data.

Method	Accuracy	Precision	Recall	F-measure
(A) MCN-SLAM (proposed): pix2pix trained with HouseExpo	0.5831	0.8613	0.3762	0.4939
(B) MCN-SLAM (proposed): U-net trained with HouseExpo	0.5852	0.8612	0.3807	0.4963
(C) Pseudo SLAM	0.4340	0.8517	0.0616	0.1110
(D) Baseline	0.4893	0.8026	0.1753	0.2734
(A) MCN-SLAM (proposed): pix2pix trained with HOME'S	0.5065	0.8537	0.2222	0.3481
(B) MCN-SLAM (proposed): U-net trained with HOME'S	0.5257	0.7915	0.2976	0.4286

Note: The top two columns show the experimental results using HouseExpo for the training data, and the bottom two columns show the experimental results using HOME'S for the training data.

Figure 6. Experiment 2: example of Gazebo simulation environment used in the experiment.

Table 3. Experiment 2: experimental results of map completion in Gazebo simulation.

Method	Accuracy	Precision	Recall	F-measure
(A) MCN-SLAM (proposed): pix2pix	0.7864	0.4486	0.7072	0.4858
(B) MCN-SLAM (proposed): U-net	0.8028	0.4686	0.7561	0.5275
(C) Gmapping	0.8314	0.6836	0.0994	0.1671
(D) Baseline	0.8429	0.7765	0.3371	0.4062
(A) MCN-SLAM (proposed): pix2pix $(T-5)$	0.8715	0.6608	0.6962	0.6546
(B) MCN-SLAM (proposed): U-net $(T-5)$	0.8718	0.6306	0.7063	0.6429

Note: The top two columns show the experimental results of proposed methods using the HouseExpo dataset for the training data. The bottom two columns show the experimental results of proposed methods using the HouseExpo dataset at T−5 iteration.

Figure 7. Experiment 2: comparison of the maps generated using pix2pix and U-net as conditions at 1 and T−5 iterations of SLAM. This comparison shows that the loss by the discriminator is useful to improve the accuracy of map completion when the observation increases and the area of map completion is narrowed. The maps of condition show the results of Gmapping.

Figure A1. Real environment used in the experiment 3.

Figure A2. Experiment 3: example of an environment map generated in a real environment by both the proposed and comparison methods.

Table A1. Experiment 3: results of the experiment in a real environment.

Method	Accuracy	Precision	Recall	F-measure
(A) MCN-SLAM (proposed): pix2pix	0.7021	0.7582	0.2507	0.3769
(B) MCN-SLAM (proposed): U-net	0.7002	0.5985	0.3119	0.4101
(C) Gmapping	0.6711	0.9817	0.1056	0.1907
(D) Baseline	0.6802	0.7351	0.1945	0.3076

Note: The top two columns show the experimental results of proposed methods using HouseExpo as training data.