Spark-based Parallel OS-ELM Algorithm Application for Short-term Load Forecasting for Massive User Data

Abstract

The data type and quantity of user load data show an exponential growth, so that the traditional load forecasting methods can hardly meet the load forecasting requirements of massive users. Aiming at this problem, a parallel OS-ELM short-term load forecasting model based on Spark is proposed in this article. By analyzing the characteristics of the Spark framework and the MapReduce framework, the Spark big data processing framework is determined as the basic framework for processing massive user load data, and a parallel K-means load clustering model based on Spark is designed. The on-line sequential learning machine OS-ELM makes the hidden layer data of computing each incremental training dataset mutually independent, therefore, a Spark-based parallel OS-ELM (SBPOS-ELM) algorithm is put forward. The proposed model is applied under the smart electricity big data environment and the training samples are selected using the incremental training dataset to make a short-term prediction of the millions of users’ smart meter electricity load, which verifies the feasibility and effectiveness of the proposed model. At last, comparing with other commonly used short-term load forecasting algorithms, the experimental results show that SBPOS-ELM algorithm has higher accuracy and operation efficiency.

Keywords:

• parallel OS-ELM algorithm
• short-term load forecast
• massive user load data
• K-means algorithm
• Spark

Additional information

Notes on contributors

Yuancheng Li

Yuancheng Li, received the Ph.D. degree from University of Science and Technology of China, Hefei, China, in 2003. From 2004 to 2005, he was a postdoctoral research fellow in the Digital Media Lab, Beihang University, Beijing, China. Since 2005, he has been with the North China Electric Power University, where he is a professor and the Dean of the Institute of Smart Grid and Information Security. From 2009 to 2010, he was a postdoctoral research fellow in the Cyber Security Lab, college of information science and technology of Pennsylvania State University, Pennsylvania, USA. He has hosted and participated in several research projects for the National Natural Science Foundation of China, National 863 Plan projects. He is the author of more than 90 articles, and more than ten inventions. His research interests include power grid security, information security, cloud computing, big data security, and cloud security.

Rongyan Yang

Rongyan Yang is a Master degree candidate at the North China Electric Power University, Beijing, China. Her research interest is power grid security and information security.

Panpan Guo

Panpan Guois a Master degree candidate in North China Electric Power University, Beijing, China. His research interest is short-term load forecasting.