Neural architectures for gender detection and speaker identification

Figures & data

Figure 1. An original audio signal

Figure 2. MFCC feature of the audio signal

Table 1. Data sets for gender detection

Data sets	#Male	# Female	#Total
Train	488	407	855
Test	302	268	570

Table 2. Data sets for speaker identification. The number of each speaker’s audios are shown below at corresponding each Speaker ID

Data sets	Speaker ID	#Total
	1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Train	63 63 56 57 58 60 63 66 59 59 63 61 65 61 60 57 54 59 56	1140
Test	12 12 19 18 17 15 12 9 16 16 12 14 10 14 15 18 21 16 19	285

Table 3. Hyper-parameters of MLP used for both tasks

MLP	Small			Large
Layers	Layer-1	Layer-2	Layer-3	Layer-1	Layer-2	Layer-3
Hidden units	128	64	32	512	256	128
Dropout	0.15	0.15	0.15	0.15	0.15	0.15
Activation function	Relu	Relu	Relu	Relu	Relu	Relu

Table 4. Hyper-parameters of CNN used for both tasks

CNN	Small			Large
Layers	Conv. maxP	Conv. maxP	Dense	Conv. maxP	Conv. maxP	Dense
Hidden units	128	64	32	512	256	128
Dropout	0.15	0.15	0.15	0.15	0.15	0.15
Activation function	Relu	Relu	Relu	Relu	Relu	Relu

Figure 3. Distribution of audios for gender and speakers

Figure 4. Training curve of MLP and CNN for gender detection

Table 5. Results for gender recognition with different features: L denotes for using the normalized fattened long mfcc vector; G denotes for using the z-score and Gramian matrix transformation. P is precision, R is recall, and F1 is F1-score

Models		Male			Female			Macro, avg
		P	R	F1	P	R	F1	P	R	F1
L	MLP-small	79.56	85.09	82.24	81.78	75.37	78.44	80.67	80.23	80.34
	MLP-large	77.22	92.05	83.98	88.57	69.40	77.82	82.89	80.72	80.90
G	MLP-small	100	100	100	100	100	100	100	100	100
	MLP-large	100	100	100	100	100	100	100	100	100
L	CNN-small	79.68	83.11	81.36	80	76.11	78.01	80.55	79.07	79.21
	CNN-large	76.57	88.74	82.20	84.54	69.40	76.22	79.84	79.61	79.68
G	CNN-small	100	100	100	100	100	100	100	100	100
	CNN-large	100	100	100	100	100	100	100	100	100

Table 6. Results for speaker identification

Models		Macro, avg.
		P	R	F1
L	MLP-small	20.76	20.36	19.94
	MLP-large	21.76	21.92	21.50
G	MLP-small	98.66	98.57	98.58
	MLP-large	99.36	99.28	99.30
L	CNN-small	36.78	35.68	35.25
	CNN-large	36.77	36.51	36.16
G	CNN-small	99.17	99.09	99.12
	CNN-large	99.17	99.47	99.47

Figure 5. Training curve of MLP and CNN for speaker identification

Figure 6. Results for gender detection after adding the noise signal to the test set

Figure 7. Results for speaker identification after adding the noise signal to the test set

Figure 8. T-SNE visualization of the intermediate output of MLP and CNN models for test set after model training