Abstract
The container pre-marshalling problem (CPMP) aims to minimise the number of reshuffling moves, ultimately achieving an optimised stacking arrangement in each bay based on the priority of containers during the non-loading phase. Given the sequential decision nature, we formulated the CPMP as a Markov decision process (MDP) model to account for the specific state and action of the reshuffling process. To address the challenge that the relocated container may trigger a chain effect on the subsequent reshuffling moves, this paper develops an improved policy-based Monte Carlo tree search (P-MCTS) to solve the CPMP, where eight composite reshuffling rules and modified upper confidence bounds are employed in the selection phases, and a well-designed heuristic algorithm is utilised in the simulation phases. Meanwhile, considering the effectiveness of reinforcement learning methods for solving the MDP model, an improved Q-learning is proposed as the compared method. Numerical results show that the P-MCTS outperforms all compared methods in scenarios where all containers have different priorities and scenarios where containers can share the same priority.
Acknowledgement
This research was made possible with funding support from National Natural Science Foundation of China [72101203, 71871183], Shaanxi Provincial Key R&D Program, China [2022KW-02], and China Scholarship Council [grant number 202206290124].
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
Data sharing not applicable – no new data generated.
Additional information
Funding
Notes on contributors
Ziliang Wang
Mr. Ziliang Wang, is a Doctoral student from School of Management in Northwestern Polytechnical University.
Chenhao Zhou
Dr. Chenhao Zhou, is a Professor from School of Management in Northwestern Polytechnical University. Prior to this, he was a Research Assistant Professor in the Department of Industrial Systems Engineering and Management, National University of Singapore. His research interests are transportation systems and maritime logistics using simulation and optimization methods.
Ada Che
Dr. Ada Che, is a Professor from School of Management in Northwestern Polytechnical University. He received the B.S. and Ph.D. degrees in Mechanical Engineering from Xi’an Jiaotong University in 1994 and 1999, respectively. Since 2005, he has been a Professor in School of Management in Northwestern Polytechnical University. His current research interests include transportation planning and optimisation, production scheduling, and operations research.
Jingkun Gao
Mr. Jingkun Gao, is currently an Engineer with Northwest Electric Power Design Institute Co., Ltd. of China Power Engineering Consulting Group and he received the Master’s degree from School of Management in Northwestern Polytechnical University.