Multi-agent reinforcement learning to unify order-matching and vehicle-repositioning in ride-hailing services

Abstract

The popularity of ride-hailing platforms has significantly improved travel efficiency by providing convenient and personalized transportation services. Designing an effective ride-hailing service generally needs to address two tasks: order matching that assigns orders to available vehicles and proactive vehicle repositioning that deploys idle vehicles to potentially high-demand regions. Recent studies have intensively utilized deep reinforcement learning to solve the two tasks by learning an optimal dispatching strategy. However, most of them generate actions for the two tasks independently, neglecting the interactions between the two tasks and the communications among multiple drivers. To this end, this paper provides an approach based on multi-agent deep reinforcement learning where the two tasks are modeled as a unified Markov decision process, and the colossal state space and competition among drivers are addressed. Additionally, a modifiable agent-specific state representation is proposed to facilitate knowledge transferring and improve computing efficiency. We evaluate our approach on a public taxi order dataset collected in Chengdu, China, where a variable number of simulated vehicles are tested. Experimental results show that our approach outperforms seven existing baselines, reducing passenger rejection rate, driver idle time and improving total driver income.

Keywords:

• Order matching
• vehicle repositioning
• multi-agent reinforcement learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Acknowledgements

We appreciate Professors Christophe Claramunt and May Yuan, and anonymous reviewers for their constructive comments that helped improve the quality of the paper.

Data and codes availability statement

The codes that support this work are available in GitHub at https://github.com/mingyxu/MAMR. The dataset can be downloaded from https://outreach.didichuxing.com.

Notes

1 https://outreach.didichuxing.com

2 https://h3geo.org/

Additional information

Funding

The work was supported by National Natural Science Foundation of China [No. 42071354] and DongFeng ChangXing Technology Limited Company.

Notes on contributors

Mingyue Xu

Mingyue Xu is an M.S. student in the School of Remote Sensing and Information Engineering at Wuhan University. Her research interest is geographic information systems, smart cities, and intelligent transport systems. She contributed to the idea, study design, methodology, implementation, and manuscript writing of this paper.

Peng Yue

Peng Yue is a professor at Wuhan University. He serves as the director at the Hubei Province Engineering Center for Intelligent Geoprocessing and the director at the Institute of Geospatial Information and Location Based Services. He supervised the research and contributed to this paper’s idea, study design, methodology, and manuscript writing.

Fan Yu

Fan Yu is an engineer of Dongfeng Changxing Tech. Co.Ltd. He contributed to this paper’s idea, study design, and methodology.

Can Yang

Can Yang is a postdoctoral researcher in the School of Remote Sensing and Information Engineering at Wuhan University. His research interest is in trajectory pattern mining and recognition. He contributed to this paper’s idea, study design, methodology, and manuscript writing.

Mingda Zhang

Mingda Zhang is an instructor in the School of Resources and Environment at Hubei University. His research interest is in the geographic information system. He contributed to this paper’s idea, study design, and methodology.

Shangcheng Li

Shangcheng Li is an engineer of Dongfeng Changxing Tech. Co.Ltd. He contributed to this paper’s idea, study design, and methodology.

Hao Li

Hao Li is a Ph.D. student in the School of Remote Sensing and Information Engineering at Wuhan University. His research interest is in intelligent transportation and knowledge graph. He contributed to this paper’s idea, study design, and methodology.