Spatial arrangement using deep reinforcement learning to minimise rearrangement in ship block stockyards

Figures & data

Figure 1. Block arrangement problem and operational processes in shipyards.

Figure 2. Decision processes in block stockyards arrangement problem.

Table 1. Definitions of reinforcement learning components pertaining to the LA and TA.

Agent	State	Action	Reward
LA	Blocks in stockyard, including location and remaining days information	Selecting the arrangement location of the next block	Number of blocks moved during the rearrangement stage
TA	Blocks and transporter in stockyard	Moving a transporter or carrying a block on a transporter	Success or failure of exporting a block

Figure 3. Examples of (a) a grid and (b) rearrangement problem.

Figure 4. LA training process based on the A3C approach.

Figure 5. Geographical information and schematic drawing of the target stockyard.

Figure 6. Episode length and reward during a training TA for 200,000 episodes.

Figure 7. Failure rate and average number of moved blocks according to area occupancy ratio.

Figure 8. Examples of rearrangement process using the trained TA.

Figure 9. Histograms and probability distribution functions of arrangement period and interarrival time.

Figure 10. Episode length and reward during a training LA for 11,000 episodes.

Figure 11. Example policy for the trained LA.

Figure 12. Schedule for a single block stockyard.

Figure 13. Comparison of rearrangement results obtained using (a) the A3C algorithm and the record data, (b) the A3C and BLF algorithms and (c) the A3C and PSLAP algorithms.

Table 2. Number of rearrangement for heuristic algorithms and A3C algorithm.

		Number of rearrangement
Algorithm		$5\times 5$ cells	$5\times 10$ cells
Heuristic	BLF	48	132
	PSLAP	47	114
Reinforcement Learning	A3C	41	96