L-measure evaluation metric for fake information detection models with binary class imbalance

ABSTRACT

Fake information in social media frequently causes social issues. The amount of fake information is smaller than that of real information, this leads to class imbalance. Some improved classification methods and metrics to resolve the imbalance and evaluate model performance have been proposed, respectively. However, the existing metrics for classification methods have many limitations. This paper proposes the robust metric, L-measure, that can reasonably evaluate all models with binary class imbalance with different IRs. L-measure also require less computation than the Matthews correlation coefficient. Finally, this paper demonstrates the validity of the proposed metric under different IRs with examples from UCI and Kaggle.

KEYWORDS:

• Class imbalance problem
• model evaluation
• binary classification
• F₁-measure
• Matthews correlation coefficient
• fake information

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was supported by National Key R&D Program of China, No. 2018YFE0105000, 2018YFB1305304, the Shanghai Municipal Commission of Science and Technology No. 19511132100, and the National Natural Science Foundation of China under Grant No. 51475334.