Opinion mining: a hybrid framework based on lexicon and machine learning approaches

ABSTRACT

Sentiment analysis is a practical technique that allows businesses, researchers, governments, politicians, and organizations to know about people's sentiments, which play an important role in decision-making processes. Sentiment classification techniques are mainly divided into lexicon-based methods, machine learning methods, and hybrid methods. There are limitations in each approach; Traditional machine learning approaches are based on complex features extraction process, and lexicon-based approaches suffer from scalability and are limited by unreliable sentiment lexicons that are commonly created manually by experts. In this paper, we seek to improve the performance of machine learning techniques by integrating it with our enhanced lexicon based technique Sum-of-Votes Model. Sum-of-Votes model is a generic extendable lexicon based model that beats traditional lexicon based models in accuracy and provides good solutions to previous challenges and drawbacks such as scalability, domain dependency, and unreliability, but its accuracy was 81.62%. So, in this paper we proposed a novel framework based upon both Sum-of-Votes and Bag-of-Words models; we applied them, then their outputs were fed as features to Machine Learning Classifiers. We got higher accuracy than all the individual lexicons and the entire old framework.

KEYWORDS:

• Natural language processing
• sentiment analysis
• hybrid technique
• machine learning

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 http://www.text-utils.com/remove-duplicate-lines.

2 http://scikit-learn.org.

Additional information

Notes on contributors

Mohammed Elsaid Moussa

Mohammed Elsaid Moussa is a Teacher Assistant in Computer Science department, Faculty Of Computers and Information, Helwan University, Cairo, Egypt. He holds a bechelor degree in computer science with excellent grade, and a Master degree in computer Science from the faculty of computers and information, Helwan University, Cairo, Egypt.

Ensaf Hussein Mohamed

Dr. Ensaf Hussein Mohamed received her Ph.D. in computer science, a faculty of computers and information, Helwan University, Cairo, Egypt, 2013. Her recent research focuses on Natural Language Processing, Text Mining, Machine Learning. Currently, she is an assistant professor, a faculty of computers and information, Helwan University, Cairo, Egypt.

Mohamed Hassan Haggag

Professor Mohamed Hassan Haggag is a Professor of Computer Science, Faculty Of Computers and Information, Helwan University, Cairo, Egypt. Haggag is the President of the Scientific Account Center, Helwan University, Cairo, Egypt.