Improving accuracy of host load predictions on computational grids by artificial neural networks

Abstract

The capability to predict the host load of a system is significant for computational grids to make efficient use of shared resources. This work attempts to improve the accuracy of host load predictions by applying a neural network predictor to reach the goal of best performance and load balance. We describe the feasibility of the proposed predictor in a dynamic environment, and perform experimental evaluation using collected load traces. The results show that the neural network achieves consistent performance improvement with surprisingly low overhead in most cases. Compared with the best previously proposed method, our typical 20:10:1 network reduces the mean of prediction errors by approximately up to 79%. The training and testing time is extremely low, as this network needs only a couple of seconds to be trained with more than 100,000 samples, in order to make tens of thousands of accurate predictions within just a second.

Keywords:

• host load
• neural networks
• predictor
• grid computing
• scheduling

Acknowledgements

We would like to thank Dinda and Yang for the use of their collected load traces. Thanks are also due to the anonymous reviewers for their valuable comments to help improve this work. This research has been partially supported by Ministry of Education, Culture, Sports, Science and Technology of Japan under the ‘21st Century COE Program’.