The impact of sociality regimes on heterogeneous cooperative-competitive multi-agent reinforcement learning: a study with the predator-prey game

References

• Anastassacos, N., Hailes, S., & Musolesi, M. (2020). Partner selection for the emergence of cooperation in multi-agent systems using reinforcement learning. In Proceedings of The AAAI Conference On Artificial Intelligence, New York, USA (Vol. 34). pp. 7047–7054.
   Google Scholar
• Barnes, S. (2022, 2). Decentralized multi-agent advantage actor-critic. https://doi.org/10.36227/techrxiv.19166384
   Google Scholar
• Brainov, S. (1999). The role and the impact of preferences on multiagent interaction. In International Workshop On Agent Theories, Architectures and Languages, Orlando, USA (pp. 349–363).
   Google Scholar
• Burnell, R., Schellaert, W., Burden, J., Ullman, T. D., Martinez-Plumed, F., Tenenbaum, J. B., Rutar, D., Cheke, L. G., Sohl-Dickstein, J., Mitchell, M., Kiela, D., Shanahan, M., Voorhees, E. M., Cohn, A. G., Leibo, J. Z., & Hernandez-Orallo, J. (2023). Rethink reporting of evaluation results in ai. Science, 380(6641), 136–138. https://doi.org/10.1126/science.adf6369
   PubMed Web of Science ®Google Scholar
• Cao, K., Lazaridou, A., Lanctot, M., Leibo, J. Z., Tuyls, K., & Clark, S. (2018). Emergent communication through negotiation. International conference on learning representations, Vancouver, Canada.
   Google Scholar
• Eckel, C. C., & Grossman, P. J. (1996). Altruism in anonymous dictator games. Games and Economic Behavior, 16(2), 181–191. https://doi.org/10.1006/game.1996.0081
   Web of Science ®Google Scholar
• Fehr, E., & Fischbacher, U. (2002). Why social preferences matter–the impact of non-selfish motives on competition, cooperation and incentives. The Economic Journal, 112(478), C1–C33. https://doi.org/10.1111/1468-0297.00027
   Web of Science ®Google Scholar
• Fehr, E., & Schmidt, K. M. (1999). A theory of fairness, competition, and cooperation. The Quarterly Journal of Economics, 114(3), 817–868. https://doi.org/10.1162/003355399556151
   Web of Science ®Google Scholar
• Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675–701.
   Google Scholar
• Gower-Winter, B., & Nitschke, G. (2022). Do harsher environments cause selfish or altruistic behavior? Proceedings of the genetic and evolutionary computation conference companion, Boston, USA (pp. 144–147).
   Google Scholar
• Grupen, N. A., Selman, B., & Lee, D. D. (2022). Cooperative multi-agent fairness and equivariant policies. Proceedings of the aaai conference on artificial intelligence, Vancouver, Canada (Vol. 36. pp. 9350–9359).
   Google Scholar
• Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6), 750–797. https://doi.org/10.1007/s10458-019-09421-1
   Web of Science ®Google Scholar
• Hogg, L., & Jennings, N. R. (1999). Variable sociability in agent-based decision making. International workshop on agent theories, architectures, and languages, Orlando, USA (pp. 305–318).
   Google Scholar
• Ivanov, D., Egorov, V., & Shpilman, A. (2021). Balancing rational and other-regarding preferences in cooperative-competitive environments. Proceedings of the 20th international conference on autonomous agents and multiagent systems, London, UK (pp. 1536–1538).
   Google Scholar
• Leibo, J. Z., Duen˜ez-Guzman, E. A., Vezhnevets, A., Agapiou, J. P., Sunehag, P., Koster, R., & Graepel, T. (2021). Scalable evaluation of multi-agent reinforcement learning with melting pot. International conference on machine learning (pp. 6187–6199).
   Google Scholar
• Lei, W., Wen, H., Wu, J., & Hou, W. (2021). Maddpg-based security situational awareness for smart grid with intelligent edge. Applied Sciences, 11(7), 3101. https://doi.org/10.3390/app11073101
   Google Scholar
• Li, Y. L. L., Liang, L., Fu, J., Wang, J., & Yin, Y. (2022). Multiagent reinforcement learning for task offloading of space/aerial-assisted edge computing. Security and Communication Networks, 2022, 1–10. https://doi.org/10.1155/2022/4193365
   Web of Science ®Google Scholar
• Lillicrap, T. L. A. I. L. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. https://doi.org/10.1016/S1098-3015(10)67722-4
   Google Scholar
• Li, S., Wu, Y., Cui, X., Dong, H., Fang, F., & Russell, S. (2019). Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient. Proceedings of the aaai conference on artificial intelligence, Honolulu, USA (Vol. 33. pp. 4213–4220).
   Google Scholar
• Lowe, R., Wu, Y. I., Tamar, A., Harb, J., Pieter Abbeel, O., & Mordatch, I. (2017). Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in Neural Information Processing Systems, 30, 30. https://doi.org/10.48550/arXiv.1706.02275
   Google Scholar
• Mashyekhi, M., Ajmeri, N., List, G. F., & Singh, M. P. (2022). Prosocial norm emergence in multi-agent systems. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 17(1–2), 1–24. https://doi.org/10.1145/3540202
   Web of Science ®Google Scholar
• McDonald, C. (2022, 8). Sufficient social reasoning to learn productive cycles of cooperation. https://doi.org/10.1184/R1/20492916.v1
   Google Scholar
• Mordatch, I., & Abbeel, P. (2018). Emergence of grounded compositional language in multiagent populations. Proceedings of the aaai conference on artificial intelligence, New Orleans, USA (Vol. 32).
   Google Scholar
• Peysakhovich, A., & Lerer, A. (2018a). Consequentialist conditional cooperation in social dilemmas with imperfect information (short workshop version). Workshops at the Thirty-Second Aaai Conference on Artificial Intelligence.
   Google Scholar
• Peysakhovich, A., & Lerer, A. (2018b). Prosocial learning agents solve generalized stag hunts better than selfish ones. Proceedings of the 17th international conference on autonomous agents and multiagent systems, Stockholm, Sweden (pp. 2043–2044).
   Google Scholar
• Powell, L. J. (2022). Adopted utility calculus: Origins of a concept of social affiliation. Perspectives on Psychological Science, 17(5), 1215–1233. https://doi.org/10.1177/17456916211048487
   Web of Science ®Google Scholar
• R˘adulescu, R., Mannion, P., Roijers, D. M., & Nowé, A. (2020). Multi-objective multi-agent decision making: A utility-based analysis and survey. Autonomous Agents and Multi-Agent Systems, 34(1), 10. https://doi.org/10.1007/s10458-019-09433-x
   Web of Science ®Google Scholar
• Ryu, H., Shin, H., & Park, J. (2020). Multi-agent actor-critic with hierarchical graph attention network. Proceedings of the aaai conference on artificial intelligence, New York, USA (Vol. 34. pp. 7236–7243).
   Google Scholar
• Savage, L. J. (1972). The foundations of statistics. Courier Corporation.
   Google Scholar
• Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. https://doi.org/10.48550/arXiv.1707.06347
   Google Scholar
• Schwarting, W. (2021). Learning and control for interactions in mixed human-robot environments [ Unpublished doctoral dissertation]. Massachusetts Institute of Technology.
   Google Scholar
• Schwarting, W., Pierson, A., Alonso-Mora, J., Karaman, S., & Rus, D. (2019). Social behavior for autonomous vehicles. Proceedings of the National Academy of Sciences, 116(50), 24972–24978. https://doi.org/10.1073/pnas.1820676116
   PubMed Web of Science ®Google Scholar
• Soubeyran, R. (2021). Pro-social motivations, externalities and incentives. Working Papers.
   Google Scholar
• Sunder, V., Vig, L., Chatterjee, A., & Shroff, G. (2018). Prosocial or selfish? agents with different behaviors for contract negotiation using reinforcement learning. International workshop on agent-based complex automated negotiation, Stockholm, Sweden (pp. 63–81).
   Google Scholar
• Wang, W., Hao, J., Wang, Y., & Taylor, M. (2019). Achieving cooperation through deep multiagent reinforcement learning in sequential prisoner’s dilemmas. Proceedings of the first international conference on distributed artificial intelligence, Singapore (pp. 1–7).
   Google Scholar
• Wang, Y., & Wu, F. (2020). Policy adaptive multi-agent deep deterministic policy gradient. International conference on principles and practice of multi-agent systems, Nagoya, Japan (pp. 165–181).
   Google Scholar
• Yi, Y., Li, G., Wang, Y., & Lu, Z. (2022). Learning to share in networked multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 35, 15119–15131. https://doi.org/10.48550/arXiv.2112.08702
   Google Scholar
• Zhang, K., Yang, Z., & Ba¸sar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, 321–384. https://doi.org/10.1007/978-3-030-60990-0_12
   Google Scholar