A lightweight and style-robust neural network for autonomous driving in end side devices

Figures & data

Figure 1. Example images captured from various weathers in CARLA. The quality and clarity of these images are different, thus influencing the performance of the driving models. (a) clear noon, (b) clear noon after rain, (c) heavy rain noon, (d) clear sunset, (e) after rain sunset, (f) soft rain sunset.

Table 1. The driving results of the CILRS (Codevilla et al., 2019) agent in different weather conditions.

Tasks	Clear sunset	Clear noon	After rain noon	Heavy rain noon
Task1	0.96	1.00	0.88	0.96
Task2	0.72	0.84	0.68	0.76
Task3	0.28	0.40	0.52	0.24

Figure 2. The mapping network. The output style codes i.e. clear noon, clear noon after rain, heavy rain noon, and clear sunset, are the input of the generator.

Figure 3. The inputs and outputs of the driving model.

Figure 4. The workflow of Star-CILRS. The proposed method contains two main models: the translation network and the driving model. The translation network takes a source image as input and outputs a generated image with a specific weather style. Given the generated image, the driving model takes actions for autonomous driving.

Figure 5. The inference stage of our method. The generator takes a source image and a target style code to output the target image with a specific style. Then, the translated image is fed to the driving model to control the agent.

Figure 6. Town01 and Town02 in CARLA (Dosovitskiy et al., 2017).

Figure 7. Example images generated by the translation network in empty scenarios. Source images are in the first row and reference images are in the first column where all images are real images. The rest images in the middle are the generated images with the style extracted from the images in the first column. These images are collected in simple task scenarios with only a few vehicles.

Figure 8. Example images generated with vehicles. Source images with vehicles are in the first row, and reference images are in the first column. The rest of the images in the middle are the generated images.

Table 2. Comparison with the state-of-the-arts on the CARLA benchmark in Town02 with training weathers.

Task	CIL	CAL	MT	CILRS	ours
task1	48±3	36±6	41±2	66±2	74±2
task2	27±1	26±2	22±0	49±5	40±5
task3	10±2	9±1	7±1	23±1	7±2

Table 3. Comparison with the state-of-the-art on the CARLA benchmark in Town01 with testing weathers.

Task	CIL	CAL	MT	CILRS	ours
task1	83±2	85±2	58±2	96±1	100±2
task2	55±5	68±5	40±6	77±1	75±3
task3	13±4	33±2	7±2	47±5	24±8

Table 4. Comparison with the state-of-the-art on the CARLA benchmark in Town02 with testing weathers.

Task	CIL	CAL	MT	CILRS	OURS(clear sunset)	OURS(clear noon)
task1	24±1	25±3	57±0	90±2	98±2	98
task2	13±2	14±2	32±2	56±2	74±2	48
task3	2±0	10±0	14±2	24±8	22±2	12

Codevilla, F., Santana, E., López, A. M., & Gaidon, A. (2019). Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 9329–9338). IEEE.

 Google Scholar

Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017, November). CARLA: An open urban driving simulator. In S. Levine, V. Vanhoucke, & K. Goldberg (Eds.), Proceedings of the 1st annual conference on robot learning (Vol. 78, pp. 1–16). PMLR. http://proceedings.mlr.press/v78/dosovitskiy17a.html.

 Google Scholar