On the Influence of Class Noise in Medical Data Classification: Treatment Using Noise Filtering Methods

Figures & data

Figure 1. Dataset with two examples with class noise. Figure 1a shows the exemplary decision boundary created when considering the noisy samples as proper ones. Figure 1b shows a decision boundary after the removal of the noisy examples during the filtering. The decision boundary obtained after the filtering is less complex and leads to a better generalization of these two classes.

Table 1. Parameter setup for the noise filters.

Noise filter	Acronym	Parameters
ensemble filter	EF	voting scheme: majority, n: 3
edited nearest neighbor	ENN	k-value: 3, distance: HVDM
iterative-partitioning filter	IPF	voting scheme: majority, n: 3
multiedit	ME	k-value: 1, subblocks: 3, distance: HVDM
nearest centroid neighborhood edition	NCNE	k value: 3
relative neighborhood graph	RNG	order of the graph: first, distance: HVDM

Table 2. Medical datasets considered in the experimentation.

Dataset	#C	#E	#A	Dataset	#C	#E	#A
breast tissue	6	106	9 (9/0/0)	mice protein	8	552	77 (77/0/0)
breast cancer	3	675	84 (84/0/0)	Parkinson’s	2	195	22 (22/0/0)
fertility	2	100	9 (2/0/7)	postoperative	3	87	8 (0/1/7)
hypertension	6	1183	18 (3/7/8)	retinopathy	2	1151	19 (16/0/3)
inflammations	2	120	6 (1/0/5)	statlog	2	270	13 (7/0/6)
kidney disease	2	158	24 (6/5/13)	thoracic surgery	2	470	16 (3/0/13)

Table 3. Parameter setup for the classification algorithms.

Classifier	Parameters
C4.5	Confidence: 0.25, minimal instances per leaf: 2, prune after the tree building
SVM	Kernel: polynomial, exponent: 1, C: 1.0, tolerance: 0.001, ε = 10^–12
NN	Distance: HVDM

Table 4. Accuracies (%) obtained by C4.5 with respect to a given class noise level and noise filtering method.

Noise	0	5	10	20	30	0	5	10	20	30	0	5	10	20	30
Dataset	breast tissue					breast cancer					fertility
None	65.11	59.44	54.72	65.97	56.62	89.63	88.89	82.96	76.44	74.22	88	88	88	89	88
EF	68.83	65.02	66.97	69.87	64.11	90.07	90.22	89.19	88	87.41	88	88	88	88	88
ENN	55.71	51.08	46.19	46.19	48.18	90.07	85.93	86.96	84.44	82.07	88	88	88	85	88
IPF	64.03	60.48	65.11	66.93	64.07	89.78	92	90.52	90.52	84.74	88	88	88	88	88
ME	35.8	36.75	31.99	34.85	33.9	82.37	77.63	78.37	77.93	74.37	88	88	88	88	88
NCNE	62.34	66.97	65.11	65.11	61.3	89.93	88.59	90.37	90.37	84.74	88	88	88	88	88
RNG	55.67	53.72	53.64	54.68	52.86	89.04	88	86.07	84.15	80.74	88	88	88	88	88
Dataset	hypertension					inflammations					Kidney disease
None	75.32	76.58	73.96	72.78	68.47	100	100	99.17	98.33	94.17	97.48	97.48	97.48	89.92	94.94
EF	78.19	78.7	78.7	78.11	79.04	100	100	100	100	96.67	97.48	97.48	98.75	97.48	92.48
ENN	78.11	78.45	77.35	77.01	77.6	100	100	100	100	95	90.5	91.13	82.92	82.28	82.92
IPF	77.09	78.87	78.61	77.34	78.87	100	100	100	100	96.67	98.1	96.85	98.1	96.83	96.85
ME	77.09	77.26	77.09	77.09	77.35	96.67	96.67	96.67	96.67	86.67	89.33	91.05	84.88	81.67	80.42
NCNE	78.02	78.1	78.61	77.35	77.18	100	100	100	100	95	97.48	97.48	97.48	97.48	96.83
RNG	77.18	77.52	77.43	77.51	77.94	100	100	100	98.33	92.5	96.21	96.21	96.21	95.54	88.06
Dataset	Mice protein					Parkinson’s					postoperative
None	89.32	88.78	89.14	77.16	67.93	87.18	82.56	85.64	76.41	76.92	70.13	70.13	72.48	68.95	71.24
EF	87.87	86.05	86.78	84.42	80.42	85.64	85.64	86.15	84.1	84.1	71.24	71.24	71.24	70.13	71.24
ENN	86.42	83.89	86.05	84.24	80.98	82.05	81.54	83.59	84.62	82.56	71.24	71.24	72.42	67.91	71.24
IPF	86.06	83.16	84.43	79.73	84.79	82.05	82.56	84.62	83.08	78.46	71.24	71.24	72.42	70.13	71.24
ME	77.9	77.17	75.91	69.93	73.55	83.08	83.08	81.03	81.54	77.95	71.24	68.89	60.92	71.24	65.36
NCNE	89.32	86.96	88.78	85.7	84.98	83.08	84.1	84.1	84.1	82.56	70.13	70.13	71.31	71.31	71.24
RNG	88.76	85.88	85.87	85.32	77	86.67	80	82.05	82.05	78.97	71.24	71.24	72.42	71.24	71.24
Dataset	retinopathy					statlog					thoracic surgery
None	62.12	64.21	63.42	61.86	60.29	77.78	75.93	77.41	74.07	72.22	85.11	84.89	84.68	84.89	83.83
EF	66.38	66.55	65.77	64.99	65.51	76.3	78.52	75.19	77.04	76.3	85.11	85.11	85.11	85.11	85.11
ENN	63.51	64.9	62.99	64.46	63.25	74.81	73.33	72.22	73.7	71.11	85.11	85.11	84.47	85.11	85.11
IPF	63.34	66.12	64.29	64.38	61.77	78.89	79.63	79.63	78.15	75.19	85.11	85.11	85.11	84.26	85.11
ME	62.12	63.51	62.38	64.12	63.86	67.04	66.67	68.52	67.41	68.89	85.11	85.11	84.89	85.11	83.4
NCNE	66.29	64.46	66.2	62.3	62.73	78.89	78.15	80.37	79.26	76.67	85.11	85.11	84.89	85.11	83.19
RNG	64.9	61.86	63.42	63.86	64.55	71.48	71.11	71.85	72.59	68.89	85.11	85.11	85.11	85.11	85.11

Table 5. Accuracies (%) obtained by SVM with respect to a given class noise level and noise filtering method.

Noise	0	5	10	20	30	0	5	10	20	30	0	5	10	20	30
Dataset	breast tissue					breast cancer					fertility
None	60.35	61.3	57.58	57.58	51.9	97.04	97.19	96	96.15	94.81	88	88	88	83	84
EF	62.25	61.21	57.53	51.86	52.86	96.59	96.74	96.59	97.04	95.7	88	88	88	85	87
ENN	43.38	45.28	42.42	42.34	41.47	95.56	96	94.67	95.26	94.81	88	88	87	87	84
IPF	62.21	57.49	60.3	55.63	55.67	96.44	96.74	96.3	96.74	95.26	88	88	88	88	88
ME	35.76	35.76	31.95	35.76	34.85	94.22	94.22	94.52	93.93	93.78	88	88	88	88	88
NCNE	61.3	61.26	60.39	52.86	53.81	96.74	96.44	96.59	96.74	95.7	87	88	87	86	86
RNG	42.42	41.47	40.52	44.33	45.15	96.15	96.59	95.41	95.56	95.7	88	88	87	86	86
Dataset	hypertension					inflammations					kidney disease
None	79.12	78.53	78.28	78.11	77.77	100	100	100	100	100	99.38	100	97.46	95.54	97.48
EF	78.61	79.12	78.95	79.12	77.35	100	100	100	100	100	99.38	99.38	99.38	98.08	96.83
ENN	77.43	77.43	77.6	77.52	77.43	100	100	100	100	100	98.1	95.52	91.09	94.29	93.69
IPF	78.7	78.95	79.29	79.37	78.11	100	100	100	100	100	99.38	99.38	99.38	99.35	98.13
ME	77.09	77.09	77.09	77.09	77.35	96.67	96.67	96.67	96.67	87.5	98.1	95.52	90.48	91.73	86.19
NCNE	78.36	78.19	78.28	78.45	77.43	100	100	100	100	100	99.38	99.38	98.73	99.38	99.38
RNG	77.52	77.52	77.52	77.6	77.18	100	100	100	100	100	98.1	97.44	96.19	95.56	92.44
Dataset	mice protein					Parkinson’s					postoperative
None	99.27	98.91	99.46	97.64	96.56	86.67	87.69	84.1	81.03	84.62	70.13	70.13	71.37	66.73	65.49
EF	99.27	98.55	97.64	96.2	95.29	85.64	86.67	86.67	84.1	85.13	71.24	68.95	70.13	67.71	68.89
ENN	98.19	98.01	96.92	96.02	94.57	86.15	85.64	85.13	82.05	84.62	71.24	65.62	65.29	64.58	71.24
IPF	98.73	98.55	97.83	97.47	96.93	85.64	85.13	86.67	82.56	84.1	71.24	71.24	72.42	70.13	71.24
ME	91.14	90.05	90.23	86.23	80.98	84.62	84.1	84.62	83.08	81.54	71.24	71.24	61.96	67.71	65.36
NCNE	99.27	99.09	98.37	97.1	94.75	86.15	86.15	81.03	82.05	86.15	67.78	67.78	69.08	71.18	70.07
RNG	97.83	96.38	96.38	96.19	93.48	86.15	86.15	84.62	82.56	81.54	70.13	67.91	67.91	70.13	71.24
Dataset	retinopathy					statlog					thoracic surgery
None	67.59	67.07	65.94	65.25	64.12	84.44	84.07	82.59	83.7	80	84.47	84.47	84.47	84.68	84.04
EF	64.81	65.16	65.16	63.6	64.46	82.96	82.96	81.85	82.59	81.48	85.11	85.11	85.11	85.11	84.68
ENN	65.59	65.25	65.77	64.55	64.81	83.33	82.59	82.59	81.85	83.33	85.11	85.11	85.11	84.89	85.11
IPF	63.86	65.94	64.2	62.21	62.99	81.11	81.85	81.48	81.48	79.63	85.11	85.11	85.11	84.89	85.11
ME	63.68	63.42	64.12	63.86	62.99	80.74	82.96	81.11	82.22	81.85	85.11	85.11	85.11	85.11	85.11
NCNE	67.07	66.64	65.77	64.81	63.25	82.96	84.07	82.59	81.85	81.85	84.89	85.11	85.11	84.89	83.62
RNG	66.03	65.59	64.81	63.68	64.55	81.48	81.48	81.85	80.74	79.63	85.11	85.11	85.11	84.68	85.11

Table 6. Accuracies (%) obtained by NN with respect to a given class noise level and noise filtering method.

Noise	0	5	10	20	30	0	5	10	20	30	0	5	10	20	30
Dataset	breast issue					breast cancer					fertility
None	69.87	66.06	67.06	65.11	58.61	94.67	90.22	86.96	81.63	73.48	77	78	80	79	69
EF	66.15	64.2	71.65	66.1	58.57	94.67	94.37	93.78	94.07	92.74	88	86	88	83	85
ENN	53.77	55.67	55.63	45.28	55.67	91.41	90.37	90.37	88	86.96	88	86	86	84	83
IPF	66.1	64.29	67.92	67.06	61.43	94.52	94.37	94.07	93.93	92.3	88	88	88	88	88
ME	34.81	37.66	31	34.81	33.9	91.26	90.22	90.07	88.44	85.48	88	88	88	88	88
NCNE	64.2	64.2	62.34	63.16	60.48	94.67	94.52	93.93	93.63	91.7	87	88	86	83	78
RNG	57.71	58.57	54.81	61.39	51.9	92.89	92.44	90.67	88.74	87.26	88	88	88	85	85
Dataset	hypertension					inflammations					Kidney disease
None	74.64	70.08	68.64	62.64	57.06	100	98.33	99.17	90	78.33	100	98.1	92.4	89.19	81.67
EF	79.37	79.2	79.04	79.04	79.12	100	100	100	97.5	95.83	100	100	99.38	99.35	97.44
ENN	78.11	77.94	78.11	75.74	76.5	100	100	100	96.67	93.33	98.75	96.21	91.77	93.08	90.58
IPF	78.87	79.21	78.53	78.45	78.95	100	100	100	100	96.67	100	100	99.38	100	98.73
ME	77.09	77.01	77.18	77.01	77.01	96.67	96.67	96.67	96.67	86.67	85.46	89.19	86.75	83.53	82.94
NCNE	78.36	78.44	78.36	78.53	76.16	100	100	100	95.83	96.67	100	100	99.38	100	99.38
RNG	78.28	78.53	78.02	76.92	76.92	100	100	100	96.67	94.17	99.38	97.48	94.96	95.58	87.96
Dataset	mice protein					Parkinson’s					postoperative
None	98.92	95.48	91.13	80.98	72.29	93.85	93.85	84.1	80	77.44	50.52	50.39	48.24	41.44	40.07
EF	97.29	96.56	95.83	95.65	90.4	90.77	91.28	89.23	86.67	84.1	68.89	66.6	67.78	67.78	66.54
ENN	96.02	95.29	95.47	93.3	90.78	89.23	91.28	88.72	86.15	86.67	67.91	70.13	63.14	60.98	65.49
IPF	97.83	98.19	96.93	94.21	95.66	89.74	88.21	88.21	84.1	81.03	71.24	71.24	72.42	70.13	71.24
ME	89.32	86.6	85.51	79.34	73.18	85.64	85.64	86.15	84.62	81.54	71.24	71.24	61.83	68.89	71.24
NCNE	98.37	96.92	97.64	94.02	91.85	92.82	92.82	90.77	87.69	88.21	68.89	68.95	66.73	65.62	65.49
RNG	97.29	96.93	95.3	93.67	91.5	90.77	91.28	89.74	85.64	83.59	68.95	67.78	67.78	64.31	66.54
Dataset	retinopathy					statlog					thoracic surgery
None	60.38	58.91	59.26	58.56	56.91	77.78	77.41	73.7	75.19	66.67	77.02	76.17	73.83	71.91	67.45
EF	62.38	63.77	62.82	63.6	61.86	80.37	80	81.11	80.37	77.41	84.68	82.77	83.83	84.47	81.28
ENN	61.78	61.95	63.17	62.82	60.82	80	78.89	78.52	78.15	75.56	85.11	85.11	84.47	83.19	80.43
IPF	62.73	63.6	64.73	62.12	61.17	80.37	78.52	78.15	78.89	79.63	85.11	85.11	85.11	84.68	85.11
ME	62.99	62.38	63.25	62.9	61.6	77.04	76.67	77.04	75.19	70	84.89	85.11	84.04	85.11	81.28
NCNE	62.9	63.16	62.3	61.43	62.65	83.33	82.59	81.85	81.11	79.63	84.26	84.47	82.55	81.91	77.45
RNG	62.82	63.16	63.6	62.64	61.78	78.52	76.67	76.3	75.93	71.11	82.98	82.77	80.21	80.85	73.19

Table 7. Statistical comparison among classification methods without filtering.

Classifier	Rank	pFin
SVM	51.625	—
C4.5	93	1.400E-05
NN	126.875	0
pFA	2.68E-010

Table 8. Statistical comparison among noise filters and not preprocessing.

C4.5	Rank	pFin	SVM	Rank	pFin	NN	Rank	pFin
None	231.8333	1.000E-06	None	164.7833	7.840E-01	None	318.5167	0
EF	119.875	—	EF	154.4167	—	EF	138.7833	6.122E-01
ENN	252.8417	0	ENN	250.3167	3.000E-05	ENN	240.9917	1.000E-06
IPF	157.45	1.070E-01	IPF	157.6917	8.825E-01	IPF	127.55	—
ME	334.225	0	ME	318.7167	0	ME	278.175	0
NCNE	138.875	3.913E-01	NCNE	162.1	7.911E-01	NCNE	142.025	5.790E-01
RNG	238.4	0	RNG	265.475	2.000E-06	RNG	227.4583	1.000E-05
PFA	1.64E-009		pFin	1.74E-009		pFin	2.285E-009