GAN-BElectra: Enhanced Multi-class Sentiment Analysis with Limited Labeled Data

References

• Abonizio, H. Q., E. Cabrera Paraiso, and S. Barbon Junior. 2021. “Toward text data augmentation for sentiment analysis IEEE Transactions on Artifical Intelligence .” , , 1–2315 . 10.1109/tai.2021.3114390. Early Access
   Google Scholar
• Ashish, V., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. 2017. “Attention is all you need.” In Advances in Neural Information Processing Systems, 2017 December:5999–6009. Neural information processing systems foundation. https://arxiv.org/abs/1706.03762v5.
   Google Scholar
• Barriere, V., and A. Balahur. 2020. “Improving sentiment analysis over non-english tweets using multilingual transformers and automatic translation for data-augmentation,” October. http://arxiv.org/abs/2010.03486.
   Google Scholar
• Bowon, K., and H. Jin Choi. 2020. Twice fine-tuning deep neural networks for paraphrase identification. Electronics Letters 56 (9):449–50. doi:10.1049/el.2019.4183.
   Web of Science ®Google Scholar
• Brownlee, J. 2019 Statistical methods for machine learning. Discover how to transform data into knowledge with Python (Machine Learning Mastery) 291 Accessed 10 April 2022 https://machinelearningmastery.com/statistics_for_machine_learning . .
   Google Scholar
• Clark, K., M.-T. Luong, Q. V. le, and C. D. Manning. 2020. “ELECTRA: pre-training text encoders as discriminators rather than generators.” In ICLR.
   Google Scholar
• “Cloud tensor processing units (TPUs) | google cloud.” n.d. Accessed May 2, 2022. https://cloud.google.com/tpu/docs/tpus.
   Google Scholar
• “Colaboratory – Google.” n.d. Accessed May 15, 2021. https://research.google.com/colaboratory/faq.html.
   Google Scholar
• Croce, D., G. Castellucci, and R. Basili. 2020. “GAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics July 2020 Online, 2114–19. Association for Computational Linguistics.
   Google Scholar
• Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference 1 (October):4171–86.
   Google Scholar
• Edunov, S., M. Ott, M. Auli, and D. Grangier. 2018. Understanding back-translation at scale EMNLP October 31–November 4 (Association for Computational Linguistics)489–500 Brussels, Belgium.
   Google Scholar
• Edwards, A., A. Ushio, J. Camacho-Collados, H. de Ribaupierre, and A. Preece. 2021. “Guiding generative language models for data augmentation in few-shot text classification,” November. http://arxiv.org/abs/2111.09064.
   Google Scholar
• Fan, M., D. Meng, Q. Xie, L. Zina, and X. Dong. 2017 Self-Paced Co-Training International Conference on Machine Learning August 6-11, 2017 Sydney, Australia . . PMLR 2275–2284 .
   Google Scholar
• “GitHub - Crux82/Ganbert: Enhancing the BERT training with semi-supervised generative adversarial networks.” n.d. Accessed May 26, 2021. https://github.com/crux82/ganbert.
   Google Scholar
• Hemmatian, F., and M. Karim Sohrabi. 2019. A survey on classification techniques for opinion mining and sentiment analysis. Artificial Intelligence Review 52 (3):1495–545. doi:10.1007/s10462-017-9599-6.
   Web of Science ®Google Scholar
• Kim, H., J. Son, and Y.-S. Han. 2022. “LST: Lexicon-guided self-training for few-shot text classification,” February. http://arxiv.org/abs/2202.02566.
   Google Scholar
• Kingma, D. P., and B. Jimmy Lei 2015. “Adam: A method for stochastic optimization.” In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings May 7 - 9, 2015 San Diego, CA, USA, International Conference on Learning Representations. ICLR.
   Google Scholar
• Kumar, M. P., B. Packer, and D. Koller. 2010. Self-paced learning for latent variable models. Nips 1:2.
   Google Scholar
• Liddy, E. D. 2001. “Natural language processing.” https://surface.syr.edu/istpub.
   Google Scholar
• Liu, S., X. Cheng, L. Fuxin, and L. Fangtao. 2015. TASC: Topic-adaptive sentiment classification on dynamic tweets. IEEE Transactions on Knowledge and Data Engineering 27 (6):1696–709. doi:10.1109/TKDE.2014.2382600.
   Web of Science ®Google Scholar
• Loshchilov, I., and F. Hutter. 2019. “Decoupled weight Decay Regularization.” In ICLR.
   Google Scholar
• Mastoropoulou, E.-P. 2019. Enhancing Deep Active Learning Using Selective Self-Training For Image Classification (KTH ROYAL INSTITUTE OF TECHNOLOGY)Accessed 10 April 2022 https://www.diva-portal.org/smash/get/diva2:1414329/FULLTEXT01.pdf .
   Google Scholar
• “Matplotlib: Python plotting — matplotlib 3.4.2 documentation.” n.d. Accessed May 15, 2021. https://matplotlib.org/.
   Google Scholar
• Mazharul Islam, S. M., X. Dong, and G. de Melo. 2020. “Domain-specific sentiment lexicons induced from labeled documents.” In Proceedings of the 28th International Conference on Computational Linguistics Barcelona, Spain (Online) (International Committee on Computational Linguistics), 6576–87. . http://sentimentanalysis.org.
   Google Scholar
• Miao, Z., L. Yuliang, X. Wang, and W.-C. Tan. 2020. Snippext: semi-supervised opinion mining with augmented data In Proceedings of The Web Conference 2020 20-24 April 2020 617–628 Taipei, Taiwan. doi:10.1145/3366423.3380144.
   Google Scholar
• Mitchell, T. M. 1997. Machine Learning. New York: McGraw-Hill International.
   Google Scholar
• Ning, L., C.-Y. Chow, and J.-D. Zhang. 2020. SEML: A semi-supervised multi-task learning framework for aspect-based sentiment analysis. IEEE Access 8 (October):189287–97. doi:10.1109/access.2020.3031665.
   Google Scholar
• “NumPy.” n.d. Accessed May 15, 2021. https://numpy.org/.
   Google Scholar
• “Numpy.argmax — numpy v1.22 manual.” n.d. Accessed May 2, 2022. https://numpy.org/doc/stable/reference/generated/numpy.argmax.html.
   Google Scholar
• “Optimization: Stochastic gradient descent.” n.d. Accessed May 2, 2022. http://deeplearning.stanford.edu/tutorial/supervised/OptimizationStochasticGradientDescent/.
   Google Scholar
• Patel, D. . Approaches for sentiment analysis on twitter: A STATE-OF-ART STUDY Accessed 10 April 2022 https://arxiv.org/abs/1512.01043 doi:https://doi.org/10.48550/arXiv.1512.01043 arXiv:1512.01043 .
   Google Scholar
• Radford, A., W. Jeffrey, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1 (8):9. https://github.com/codelucas/newspaper.
   Google Scholar
• Ricci, F., B. Shapira, and L. Rokach. 2015. Recommender systems: Introduction and challenges. In Recommender Systems Handbook, 2nd 1 (Boston, MA: Springer) 1–34 ed. doi:10.1007/978-1-4899-7637-6_1.
   Google Scholar
• Riyadh, M., and M. Shafiq. 2021. “Towards multiclass sentiment analysis with limited labeled data.” In IEEE International Conference on Big Data December 15-18, 2022 Online, 4955–64.
   Google Scholar
• Rosenthal, S., N. Farra, and P. Nakov. 2017. “SemEval-2017 task 4: sentiment analysis in twitter.” In Proceedings of the 11th International Workshop on Semantic Evaluation ({S}em{E}val-2017), 502–18. Vancouver, Canada: Association for Computational Linguistics. 10.18653/v1/S17-2088.
   Google Scholar
• “Scipy.special.softmax — sciPy v1.8.0 manual.” n.d. Accessed May 2, 2022. https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.softmax.html.
   Google Scholar
• “Self training in semi-supervised learning — scikit-learn documentation.” n.d. Accessed November 15, 2021. https://scikit-learn.org/stable/modules/semi_supervised.html#self-training.
   Google Scholar
• Socher, R., A. Perelygin, J. Y. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. 2013. “Recursive deep models for semantic compositionality over a sentiment treebank.” In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing 18-21 October 2013 Seattle, Washington, USA, 1631–42. Association for Computational Linguistics. http://nlp.stanford.edu/.
   Google Scholar
• Sun, Y., S. Wang, L. Yukun, S. Feng, X. Chen, H. Zhang, X. Tian, D. Zhu, H. Tian, and W. Hua 2019. “ERNIE: Enhanced Representation Through Knowledge Integration.” In ArXiv E-Prints. https://github.com/PaddlePaddle/.
   Google Scholar
• “TensorFlow.” n.d. Accessed May 15, 2021. https://www.tensorflow.org/.
   Google Scholar
• “TensorFlow hub - electra large.” n.d. Accessed May 26, 2021. https://tfhub.dev/google/electra_large/2.
   Google Scholar
• “Twitter US airline sentiment | kaggle.” n.d. Accessed May 22, 2021. https://www.kaggle.com/crowdflower/twitter-airline-sentiment.
   Google Scholar
• Vries, P. G. D. 1986. Stratified random sampling. In Sampling theory for forest inventory, 31–55 doi:10.1007/978-3-642-71581-5_2. Berlin, Heidelberg: Springer.
   Google Scholar
• “What is a GPU? graphics processing units defined.” n.d. Accessed May 15, 2021. https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-gpu.html.
   Google Scholar
• Wolf, T., L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, et al. 2019. Transformers: state-of-the-art natural language processing Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations 2020 (Association for Computational Linguistics)38–45 Online. . doi:10.18653/v1/2020.emnlp-demos.6.
   Google Scholar
• “XGBoost documentation — xgboost 1.5.0-SNAPSHOT documentation.” n.d. Accessed May 16, 2021. https://xgboost.readthedocs.io/en/latest/.
   Google Scholar
• Xinzhe, L., Q. Sun, Y. Liu, S. Zheng, Q. Zhou, T.-S. Chua, and B. Schiele. 2019. Learning to self-train for semi-supervised few-shot classification. Advances in Neural Information Processing Systems 32:10276–86.
   Google Scholar
• Yang, T., H. Linmei, C. Shi, J. Houye, L. Xiaoli, and L. Nie. 2021. HGAT: Heterogeneous Graph attention networks for semi-supervised short text classification. ACM Transactions on Information Systems 39:3. doi:10.1145/3450352.
   Web of Science ®Google Scholar
• Yarowsky, D. 1995. Unsupervised word sense disambiguation rivaling supervised methods. Association for Computational Linguistics (ACL) 189–96. doi:10.3115/981658.981684.
   Google Scholar
• Zhang, L., S. Wang, and B. Liu. 2018. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8 (4):e1253.
   Web of Science ®Google Scholar