HOAX: a hyperparameter optimisation algorithm explorer for neural networks

Abstract

Computational chemistry has become an important tool to predict and understand molecular properties and reactions. Even though recent years have seen a significant growth in new algorithms and computational methods that speed up quantum chemical calculations, the bottleneck for trajectory-based methods to study photo-induced processes is still the huge number of electronic structure calculations. In this work, we present an innovative solution, in which the amount of electronic structure calculations is drastically reduced, by employing machine learning algorithms and methods borrowed from the realm of artificial intelligence. However, applying these algorithms effectively requires finding optimal hyperparameters, which remains a challenge itself. Here we present an automated user-friendly framework, HOAX, to perform the hyperparameter optimisation for neural networks, which bypasses the need for a lengthy manual process. The neural network-generated potential energy surfaces (PESs) reduce the computational costs compared to the ab initio-based PESs. We perform a comparative investigation on the performance of different hyperparameter optimiziation algorithms, namely grid search, simulated annealing, genetic algorithm, and Bayesian optimizer in finding the optimal hyperparameters necessary for constructing the well-performing neural network in order to fit the PESs of small organic molecules. Our results show that this automated toolkit not only facilitate a straightforward way to perform the hyperparameter optimisation but also the resulting neural networks-based generated PESs are in reasonable agreement with the ab initio-based PESs.

GRAPHICAL ABSTRACT

Keywords:

• Quantum chemistry
• machine learning
• neural networks
• hyperparameter optimisation

Acknowledgments

Prof. Peter Gill is one of the main contributors to the electronic structure program Q-Chem and drove the development of theoretical chemistry in his successful research carrier. We are happy to honour him with this debut of the HOAX software package. The authors thank Prof. Peter Gill, Dr. Andrew Gilbert, and E. Salazar for many inspiring and intellectually stimulating conversations and for giving an insight into the process of new method development. Additionally, the authors are thankful to Dr. Kiana Moghaddam for her help in proofreading the manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work is part of the Innovational Research Incentives Scheme Vidi 2017 with project number 016.Vidi.189.044, which is financed by the Dutch Research Council (NWO).