GAN-BElectra: Enhanced Multi-class Sentiment Analysis with Limited Labeled Data

Figures & data

Figure 1. Example of basic lexicon-based sentiment analysis.

Figure 2. GAN-BERT Architecture (Croce, Castellucci, and Basili 2020). U, L, F, G, D denote unlabeled data, labeled data, fake labels, generator, and discriminator respectively.

Figure 3. Electra-based pretrained component.

Figure 4. SG-Elect architecture (Riyadh and Shafiq 2021), U, L, C denote unlabeled data, labeled data, and combinator respectively.

Figure 5. GAN-BElectra architecture. U and L denote unlabeled and labeled data respectively.

Algorithm I: Training process of GAN-BElectra based on SG-Elect (Riyadh and Shafiq 2021)

inputs: X = XL + XU XL = xL1,xL2,…, xLn XU = xU1,xU2,…, xUm outputs: Trained model
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19	BEGIN for all x ∈ X do { CGB ← Train (GAN-BERT, X) } end for return CGB for all xU ∈ XU do { YGB ← Predict (CGB, XU) } end for return YGB for all yGB ∈ YGB do { CEL ← Train (ElectraTransformer, YGB) } end for return CEL for all xL ∈ XL do { CEL ← Train (ElectraTransformer, XL) } end for return CEL END

Figure 6. SST5 dataset composition.

Figure 7. US Airline dataset composition.

Figure 8. SemEval dataset composition.

Table 1. Summary of Results (F1-Score, Accuracy, Standard Deviation).

Dataset	Technique	Mean F1-Macro	Mean Accuracy	Standard Deviation of Mean Accuracy
SST5	GAN-BElectra	0.3796	0.3825	0.0350
	SG-Elect	0.3763	0.373	0.0249
	Electra	0.3138	0.3533	0.0225
	GAN-BERT	0.3185	0.3406	0.0337
US Airline	GAN-BElectra	0.6722	0.7108	0.0195
	SG-Elect	0.6659	0.7072	0.0195
	Electra	0.5493	0.623	0.0331
	GAN-BERT	0.6413	0.7032	0.0514
SemEval	GAN-BElectra	0.5663	0.5741	0.0145
	SG-Elect	0.558	0.5635	0.0228
	Electra	0.3992	0.4536	0.0184
	GAN-BERT	0.473	0.5208	0.0448

Table 2. Summary of Results (Standard Error, Confidence Interval of Standard Error).

Dataset	Technique	Mean Standard Error	Confidence Interval (at 95%) of Standard Error
SST5	GAN-BElectra	0.6175	0.0203
	SG-Elect	0.627	0.0202
	Electra	0.6467	0.0199
	GAN-BERT	0.6594	0.0198
US Airline	GAN-BElectra	0.2892	0.0164
	SG-Elect	0.2928	0.0165
	Electra	0.377	0.0176
	GAN-BERT	0.2968	0.0165
SemEval	GAN-BElectra	0.4259	0.0153
	SG-Elect	0.4365	0.0154
	Electra	0.5464	0.0154
	GAN-BERT	0.4792	0.0155

Table 3. GAN-BERT’s accuracy in pseudo label generation across three datasets.

Dataset	GAN-BERT Pseudo Label Accuracy
SST5	0.3328
US Airline	0.7087
SemEval	0.5209

Table 4. Detailed Results for the SST5 Dataset.

Technique	Class	Precision	Recall	F1-score	Accuracy
GAN-BElectra	Very Negative	0.3246	0.5253	0.397	0.3825
	Negative	0.4493	0.2461	0.3129
	Neutral	0.2481	0.2601	0.2537
	Positive	0.4232	0.485	0.4499
	Very Positive	0.4969	0.4773	0.4847
SG-Elect	Very Negative	0.3357	0.4854	0.3912	0.373
	Negative	0.4604	0.2755	0.3301
	Neutral	0.2617	0.3488	0.2944
	Positive	0.4434	0.3452	0.3849
	Very Positive	0.4695	0.5072	0.4807
Electra	Very Negative	0.4121	0.3831	0.2701	0.3533
	Negative	0.3828	0.2062	0.2363
	Neutral	0.2385	0.3119	0.2663
	Positive	0.3813	0.4459	0.4063
	Very Positive	0.5194	0.4514	0.3901
GAN-BERT	Very Negative	0.3114	0.4908	0.3725	0.3406
	Negative	0.4753	0.2027	0.2476
	Neutral	0.2358	0.3524	0.27
	Positive	0.3932	0.3564	0.3539
	Very Positive	0.4881	0.4159	0.4396

Table 5. Detailed Results for the US Airline Dataset.

Technique	Class	Precision	Recall	F1-score	Accuracy
GAN-BElectra	Negative	0.8896	0.7231	0.7975	0.7108
	Neutral	0.463	0.6516	0.5404
	Positive	0.6277	0.7408	0.6788
SG-Elect	Negative	0.8921	0.7202	0.7967	0.7072
	Neutral	0.4686	0.6328	0.5372
	Positive	0.5932	0.7542	0.6637
Electra	Negative	0.8282	0.6759	0.7375	0.623
	Neutral	0.4369	0.4882	0.4468
	Positive	0.398	0.5939	0.4636
GAN-BERT	Negative	0.835	0.7752	0.7938	0.7032
	Neutral	0.5015	0.5484	0.5048
	Positive	0.6342	0.6264	0.6253

Table 6. Detailed Results for the SemEval Dataset.

Technique	Class	Precision	Recall	F1-score	Accuracy
GAN-BElectra	Negative	0.4373	0.6046	0.507	0.5741
	Neutral	0.5992	0.5524	0.5727
	Positive	0.6665	0.5857	0.6191
SG-Elect	Negative	0.4374	0.6219	0.5116	0.5635
	Neutral	0.5977	0.5244	0.557
	Positive	0.639	0.5826	0.6055
Electra	Negative	0.2835	0.3515	0.3006	0.4536
	Neutral	0.5288	0.596	0.5526
	Positive	0.5434	0.3259	0.3445
GAN-BERT	Negative	0.4542	0.4787	0.44	0.5208
	Neutral	0.5736	0.5394	0.468
	Positive	0.634	0.5191	0.5109

Figure 9. Confusion matrices for SST5 dataset for (a) GAN-BElectra (b) SG-Elect (c) Electra and (d) GAN-BERT.

Figure 10. Confusion matrices (with red shades) for the US Airline dataset for (a) GAN-BElectra (b) SG-Elect (c) Electra and (d) GAN-BART and confusion matrices (with green shades) for the SemEval dataset (e) GAN-BElectra (f) SG-Elect (g) Electra and (h) GAN-BART.

Croce, D., G. Castellucci, and R. Basili. 2020. “GAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics July 2020 Online, 2114–19. Association for Computational Linguistics.

 Google Scholar

Riyadh, M., and M. Shafiq. 2021. “Towards multiclass sentiment analysis with limited labeled data.” In IEEE International Conference on Big Data December 15-18, 2022 Online, 4955–64.

 Google Scholar