Employing synthetic data for addressing the class imbalance in aspect-based sentiment classification

Figures & data

Figure 1. XML snippet of SemEval2014 restaurant Dataset.

Figure 2. Class distribution in SemEval datasets.

Table 1. Statistics of the SemEval Restaurant review datasets.

Dataset		Positive	Negative	Neutral	Total	IR
SemEval 2014	Train set	2164	807	637	3608	3.39
	Test set	728	196	196	1120	3.71
SemEval 2015	Train set	1319	489	72	1880	18
	Test set	353	207	37	597	16.13
SemEval 2016	Train set	1802	624	104	2530	17
	Test set	487	180	42	709	11.59

Figure 3. Architecture of the proposed system.

Table 2. Languages selected for backtranslation.

Language Family	Language (language code)
Indo-European	Afrikaans (af), Bengali (bn), German (de), Greek (el), Spanish (es), Persian (fa), French (fr), Gujarati (gu), Hindi (hi), Italian (it), Marathi (mr), Flemish (nl), Punjabi (pa), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Swedish (sv), Ukrainian (uk), Urdu (ur)
Austronesian	Javanese (jv), Vietnamese (vi)
Niger-Congo	Swahili (sw), Yoruba (yo), Igbo (ig)
Dravidian	Tamil (ta), Kannada (kn)
Sino-Tibetan	Chinese (zh)
Kra–Dai	Thai (th)
Turkic	Turkish (tr)
Japonic	Japanese (ja)
Koreanic	Korean (ko)
Afro-Asiatic	Arabic (ar)
Uralic	Hungarian (hu)
Altaic	Mongolian (mn)

Figure 4. Backtranslation-Example.

Figure 5. BERT model for aspect-based sentiment classification.

Figure 6. Class Distribution before and after weighted oversampling.

Figure 7. Performance of AbSC model using Weighted Oversampling.

Table 3. Experimental results.

	SemEval 2014		SemEval 2015		SemEval 2016
Class-Balancing Approach	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
None	81.72	74.68	85.11	66.17	85.77	70.56
Weighted Oversampling	80.19	75.07	85.11	66.81	87.15	71.26
CE Loss with Class Weight	81.02	75.35	83.78	67.91	85.97	72.50
Paraphrasing	81.58	76.02	84.84	69.2	87.55	73.36
Backtranslation	81.58	76.09	87.23	71.97	88.54	74.69

Figure 8. Performance of AbSC model using CE loss with class weights.

Figure 9. Performance of AbSC model using paraphrasing class balancing approach.

Figure 10. Performance of AbSC model using Backtranslation class balancing approach.

Figure 11. Performance of AbSC model using all class balancing approaches.

Figure 12. Performance (F1-Score) of AbSC model for negative class.

Figure 13. Performance (F1-Score) of AbSC model for neutral class.

Figure 14. Performance (F1-Score) of AbSC model for positive class.

Table 4. Class-wise Performance (F1-Score) of all class-balancing approaches.

Class	Class-Balancing Approach	SemEval 2014	SemEval 2015	SemEval 2016
Negative	None	77	76	77
	Weighted Oversampling	77	78	79
	CE Loss with Class Weight	77	77	77
	Paraphrasing	77	77	81
	Backtranslation	78	79	81
Neutral	None	57	31	42
	Weighted Oversampling	59	32	42
	CE Loss with Class Weight	60	37	49
	Paraphrasing	62	41	50
	Backtranslation	61	44	50
Positive	None	90	92	91
	Weighted Oversampling	88	91	92
	CE Loss with Class Weight	89	90	92
	Paraphrasing	90	91	92
	Backtranslation	90	92	93