Adaptive network traffic control with approximate dynamic programming based on a non-homogeneous Poisson demand model

ABSTRACT

In this study, we develop a stochastic dynamic traffic-flow model subject to practical restrictions under the non-homogeneous Poisson vehicle arrival process. Using the cell transmission strategy, we establish traffic dynamics between two intersections. We also discuss simulating the random demand of source links from an estimated intensity function. Additionally, we propose an algorithm to optimize time interval division for aggregated data, aiming to enhance estimation performance. We explore applying our traffic flow model to the adaptive traffic network management problem, which is formulated as a Markov decision process. Leveraging approximate dynamic programming with recursive least squares-temporal difference learning, we achieve adaptive optimal policies. To validate our approach, we conduct a series of numerical experiments with random demands. The results of non-homogeneous Poisson demand conducted using random numbers and a real-word dataset indicate high efficiency with the piecewise constant, I-SMOOTH, and MNO-PQRS estimators. Compared to the Webster and Max-pressure control systems, our proposed approximate dynamic programming-based model exhibits superior stability and applicability.

KEYWORDS:

• Non-homogeneous Poisson process
• cell transmission method
• traffic signal control
• approximate dynamic programming
• temporal difference learning

Acknowledgments

We express our gratitude to the reviewers for their insightful comments. This study is supported by the State Key Laboratory of Integrated Services Networks (Contract No. ISN25-16), Xidian University, and by Beijing Laboratoryof National Economic Security Early-warning Engineering, Beijing Jiaotong University.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 See http://www.openits.cn/openData/index.jhtml.