Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (MARLIN-OTC)

REFERENCES

• Abdulhai, B. and Kattan L. (2003) Reinforcement learn-ing: Introduction to theory and potential for transport applications. Canadian Journal of Civil Engineering, 30 (6), 981–991.
   Web of Science ®Google Scholar
• Abdulhai, B., Pringle, R., and Karakoulas, G.J. (2003) Reinforcement learning for true adaptive traffic signal control. ASCE Journal of Transportation Engineering, 129 (3), 278–285.
   Web of Science ®Google Scholar
• Abul, O., Polat, F. and Alhajj, R. (2000) Multiagent rein-forcement learning using function approximation, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Application and Reviews, 30 (4), 485–497.
   PubMed Web of Science ®Google Scholar
• Aerde, M. V., Voss, J. and McKinnon, G. (1989) INTEGRATION Simulation Model User's Guide, Queen's University.
   Google Scholar
• Akramizadeh, A., Afshar, A., and Menhaj, M.B. (2009) Multiagent reinforcement learning in extensive form games with perfect information. Journal of Applied Sciences, ISSN 1812–5654.
   Google Scholar
• Albus, J.S. (1975) A new approach to manipulator control: the cerebellar model articulation controller (CMAC), Trans. ASME, Series G. Journal of Dynamic Systems, Measurement and Control, 97,220–233.
   Google Scholar
• Alvares, I., Poznyak, A., and Malo, A. (2008) Urban traffic control problem: agame theory approach. In: CDC, 47th IEEE Conference on Decision and Control, pp. 2168–2172.
   Google Scholar
• Basar, T., and Olsder, G.J. (1999) Dynamic Noncooperative Game Theory. Classics in Applied Mathematics. 2nd edi-tion. London, U.K.
   Google Scholar
• Bazzan, A.L.C. (2005) A distributed approach for coordina-tion of traffic signal agents. Autonomous Agents and Multi-Agent Systems, 10 (2), 131–164.
   Web of Science ®Google Scholar
• Bazzan, A.L.C. (2009) Opportunities for multiagent systems and multiagent reinforcement learning in traffic control. Autonomous Agents and Multi-Agent Systems, 18 (3), 342–375.
   Web of Science ®Google Scholar
• Bingham, E. (2001) Reinforcement learning in neurofuzzy traffic signal control. European Journal of Operation Research, 131,232–241.
   Web of Science ®Google Scholar
• Boutilier, C. (1996) Planning, learning and coordination in multiagent decision processes. In: Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge, De Zeeuwse Stromen, The Netherlands, pp. 195–210.
   Google Scholar
• Brown, G.W. (1951) Iterative solutions of games by ficti-tious play. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation. Wiley, New York, pp. 374–376.
   Google Scholar
• Busoniu, L., Babuska, R., and De Schutter, B. (2008) A com-prehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man and Cybernetics, 38 (2), 156–172.
   Web of Science ®Google Scholar
• Bertsekas, DP, and Shreve, SE (1978) tochastic optimal control: the discrete time case, Mathematics in Science and Engineering, Academic Press, Inc., New York.
   Google Scholar
• Camponogara, E. and Kraus Jr, W. (2003) Distributed learning agents in urban traffic control. In: Progress in Artificial Intelligence, Proceedings of the 11th Portuguese Conference on Artificial Intelligence, pp. 324–335.
   Google Scholar
• Carmel, D. and Markovitch, S. (1996) Opponent model-ing in multi-agent systems. In: Weiss, G. and Sen, S. (eds.), Adaptation and Learning in Multi-Agent Systems, Springer-Verlag, New York, Ch. 3, pp. 40–52.
   Google Scholar
• Chalkiadakis, G. and Boutilier, C. (2004) Bayesian reinforce-ment learning for coalition formation under uncertainty. In: 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, vol. 3, pp. 1090–1097.
   Google Scholar
• Cheng, S.F., Epelman, M.A., and Smith, R.L. (2004) CoSIGN: A fictitious play algorithm for coordinated traffic signal control. INFORMS Annual Meeting, Denver, CO.
   Google Scholar
• Claus, C. and Boutilier, C. (1998) The dynamics of rein-forcement learning in cooperative multiagent systems. In: Proceedings of the 15th National Conference on Artificial Intelligence and 10th Conference on Innovative Applications of Artificial Intelligence, Madison, US, pp. 746–752.
   Google Scholar
• Filar J. and Vrieze, K. (1997) Competitive Markov decision processes. Springer-Verlag, New York.
   Google Scholar
• Fischer, F., Rovatsos, M., and Weiss, G. (2004) Hierarchical reinforcement learning in communication-mediated multiagent coordination. In: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, New York, US, pp. 1334–1335.
   Google Scholar
• Fujita, K. and Matsuo, H. (2005) Multi-agent reinforcement learning with the partly high-dimensional state space, Trans. of IEICE, J88-D-1 (4), 864–872.
   Google Scholar
• Ghods, A.H. and Rahimi-Kian, A. (2008) A game theory approach to optimal coordinated ramp metering and variable speed limits. IEEE Chinese Control and Decision Conference, Yantai, China.
   Google Scholar
• Gosavi, A. (2003) Simulation based optimization: parametric optimization techniques and reinforcement learning. Kluwer Academic, Norwell, MA.
   Google Scholar
• Hu, J. and Wellman, M.P. (2003) Nash Q-Learning for gen-eral-sum stochastic games. Journal of Machine Learning Research, 4, 1039–1069.
   Web of Science ®Google Scholar
• Huang, P. and Sycara, K. (2003) Multi-agent learning in extensive games with complete information, Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems, ACM Press, NY, USA, pp. 701–708.
   Google Scholar
• Hunt, P. B., Robertson, D. I., Bretherton, R. D. and Winton, R. I. (1981) SCOOT-a traffic responsive method of coordinating signals, Technical Report, Transport and Road Research Laboratory, Crowthorne, England.
   Google Scholar
• Jacob, C., and Abdulhai, B. (2006) Automated adaptive traffic corridor control using reinforcement learning: approach and case studies, Transportation Research Board Annual Meeting, Washington D.C.
   Google Scholar
• Kahan, J.P. and Rapoport, A. (1984) Theories of coalition formation. Lawrence Erlbaum Associates.
   Google Scholar
• Kok, J.R., Spaan, M.T.J., and Vlassis, N. (2005) Non-communicative multirobot coordination in dynamic environment. Robotics and Autonomous Systems, 50 (2/3), 99–114.
   Web of Science ®Google Scholar
• Koller, D. and Pfeffer, A. (1997) Representations and solu-tions for game theoretic problems. Artificial Intelligence, 94 (1), 167–215.
   Web of Science ®Google Scholar
• Könönen, V. (2003) Asymmetric multiagent reinforce-ment learning. In: Proceedings IEEE! WIC International Conference on Intelligent Agent Technology, Halifax, Canada, pp. 336–342.
   Google Scholar
• Kuyer, L., Whiteson, S., Bakker, B., and Vlassis, N. (2008) Multiagent reinforcement learning for urban traffic control using coordination graphs. In: Proceedings of the 19th European Conference on Machine Learning, September 2008, pp. 656–671.
   Google Scholar
• Laslier, J.F. and Walliser, B. (2005) A reinforcement learning process in extensive form games, International Journal of Game Theory, 33 (2), 219–227.
   Google Scholar
• Lazaric, A., Munoz de Cote, E., Gatti, N., and Restelli M. (2007) Reinforcement learning in extensive form games with incomplete information: the bargaining case study. In: Proceedings of the International Joint Conference on Autonomous Agents and Multi Agent Systems, Honolulu, USA, pp. 216–218.
   Google Scholar
• Li, X. and Soh, L.-K. (2004) Investigating reinforcement learning in multiagent coalition formation. Technical report no. WS-04-06, American Association of Artificial Intelligence Workshop on Forming and Maintaining Coalitions and Teams in Adaptive Multiagent Systems, pp. 22–28.
   Google Scholar
• Lin, L.( 1992 )Self-improving reactive agents based on rein-forcement learning, planning and teaching, Machine Learning, vol 8, no 3
   Google Scholar
• Littman, M. (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the 11th International Conference on Machine Learning.
   Google Scholar
• M. L. Littman (2001) Value-function reinforcement learn-ing in Markov games. The Journal of Cognitive Systems Research., 2(1), 55–66.
   Google Scholar
• Liu, Z. (2007) A survey of intelligence methods in urban traffic signal control. International Journal of Computer Science and Network Security, 7 (7), 105–112.
   Google Scholar
• Mendelson, E. (2004) Introducing Game Theory and Its Applications. CRC Press
   Google Scholar
• Mikami, S., and Kakazu, Y. (1994) Genetic reinforce-ment learning for cooperative traffic signal control. In: International Conference on Evolutionary Computation, pp. 223–228.
   Google Scholar
• Milano, P. D. (2006) Dynamic coalition formation via reinforcement learning: easing optimization problems. Master's Thesis.
   Google Scholar
• Nash, S. and Sofer, A. (1996) Linear and nonlinear program-ming. McGraw-Hill, NY.
   Google Scholar
• Oliveira, D., Bazzan, A., Silva, B., Basso, E., Nunes, L., Rossetti, R., Oliveira, E., Silva, R., and Lamb, L. (2006) Reinforcement learning-based control of traffic lights in non-stationary environments: a case study in a micro-scopic simulator. In: 4th European Workshop on Multi-Agent Systems, Lisbon, pp. 31–42.
   Google Scholar
• Osborne, M. J. (2004) An introduction to game theory. Oxford University Press, New York.
   Google Scholar
• Papageorgiou, M., Hadj-Salem, H., and Blosseville, J. M. (1991) ALINEA: A local feedback control law for on-ramp metering. Transportation Research Record: Journal of the Transportation Research Board, No. 1320, TRB, National Research Council, Washington, D.C., pp. 58–64.
   Google Scholar
• Papageorgiou M. (1998) Automatic control methods in traffic and transportation. In: Operations Research and Decision Aid Methodologies in Traffic and Transportation Management, P. Toint, M. Labbe, K. Tanczos, G. Laporte, Editors, Springer-Verlag, pp. 46–83.
   Google Scholar
• Richter, S., Aberdeen, D., and Yu, J. (2007) Natural actor-critic for road traffic optimisation. In: Advances in Neural Information Processing Systems, vol. 19, MIT Press, Cambridge.
   Google Scholar
• Salkham, A., Cunningham, R., Garg, A., and Cahill, V. (2008) A collaborative reinforcement learning approach to urban traffic control optimization. In: IEEE! WIC! ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. 2, pp. 560–566.
   Google Scholar
• Shoham, Y., Powers, R., and Grenager, T. (2003) Multi-agent reinforcement learning: A critical survey. Tech. Rep., Computer Science Dept., Stanford University, California, US.
   Google Scholar
• Sims, A. G., and Dobinson, K.W. (1980) The Sydney coor-dinated adaptive traffic (SCAT) system philosophy and benefits, Vehicular Technology, IEEE Transactions on, 1980.29(2): p. 130–137
   Web of Science ®Google Scholar
• Smith, J.M. (1982) Evolution and the Theory of Games. Cambridge Univ. Press, UK.
   Google Scholar
• Steingröver, M., Schouten, R., Peelen, S., Nijhuis, E., and Bakker, B. (2005) Reinforcement learning of traffic light controllers adapting to traffic congestion. In: BNAIC, pp. 216–223.
   Google Scholar
• Sutton, R., and Barto, A. (1998) Reinforcement learning: an introduction. MIT Press, Cambridge Mass.
   Google Scholar
• Thorpe, T.L. (1997) Vehicle traffic light control using SARSA. Master's Project Rep., Computer Science Department, Colorado State University, Fort Collins, Colorado.
   Google Scholar
• Katwijk, R.T. van, P. van Koningsbruggen, B. De Schutter and J. Hellendoorn (2005) A test bed for multi-agent control systems in road traffic management, in Applications of Agent Technology in Traffic, and Transportation (F. Klügl, A. Bazzan, and S. Ossowski, eds.), Whitestein Series in Software Agent Technologies, Basel: Birkhäuser Verlag, ISBN 3-7643-7258-3, pp. 113–131.
   Google Scholar
• Wang, X., and Sandholm, T., (2002), Reinforcement learning to play an optimal Nash equilibrium in team Markov games. In: Advanced Neural Information Processing Systems, vol. 15, pp. 1571–1578.
   Google Scholar
• Wang, Y., Diakaki, C., Kotsialos, A., and Papageorgiou, M. (2007) Towards integrated network traffic control. In: Modeling, Information, and Control of Intelligent Transportation Systems (in French), Hermes Science Publishing Ltd, UK, pp. 281–313.
   Google Scholar
• Watkins, C. and Dayan, P. (1992) Q-learning, Machine learn-ing. vol. 8, pp. 279–292.
   Google Scholar
• Weinberg, M. and Rosenschein, J.S. (2004) Best-response multiagent learning in non-stationary environments. In: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, New York, NY, pp. 506–513.
   Google Scholar
• Wen, K., Qu, S., and Zhang, Y. (2009) A machine learning method for dynamic traffic control and guidance on freeway networks, In: 2009 International Asia Conference on Informatics in Control, Automation and Robotics, pp. 67–71.
   Google Scholar
• Wiering, M.A. (2000) Learning to control traffic lights with multiagent reinforcement learning. In: 1st World Congress of the Game Theory Society Games, Utrecht, Netherlands, Basque Country University and Foundation B.B.V., Bilbao, Spain.
   Google Scholar
• Zhenlong, L. (2005) Optimal coordination of variable speed and ramp metering based on Stackelberg game. In: 12th World Congress on Intelligent Transport Systems, San Francisco, USA.
   Google Scholar