Optimising darts strategy using Markov decision processes and reinforcement learning

Abstract

This article determines an aim point selection strategy for players in order to improve their chances of winning at the classic darts game of 501. Although previous studies have considered the problem of aim point selection in order to maximise the expected score a player can achieve, few have considered the more general strategical question of minimising the expected number of turns required for a player to finish. By casting the problem as a Markov decision process and utilising the reinforcement learning method of value iteration, a framework is derived for the identification of the optimal aim point for a player in an arbitrary game scenario. This study represents the first analytical investigation of the full game under the normal game rules, and is, to our knowledge, the first application of reinforcement learning methods to the optimisation of darts strategy. The article concludes with an empirical study investigating the optimal aim points for a number of player skill levels under a range of game scenarios.

Keywords:

• Markov decision process
• reinforcement learning
• dynamic programming
• value iteration
• Monte Carlo simulation
• sports strategy

Acknowledgements

I would like to thank Dr Philip Knight for proposing this subject for my dissertation and for encouraging me to produce this paper for publication. I would also like to thank Dr Ian Griffiths for encouraging me to re-visit the project and for his support. I am also deeply grateful to Dr Kerem Akartunali for his guidance on producing this manuscript and to Philine Scheer for the many discussions and input on the algorithms used within the study. I am very thankful to the reviewers, whose suggestions improved the contents and presentation of the paper significantly.

Disclosure Statement

No potential conflict of interest was reported by the author.