Emotion recognition based on convolutional gated recurrent units with attention

Figures & data

Figure 1. An outline of the EEG-based emotion recognition framework presented with AB-DPCGRU.

Figure 2. Mapping from real 62 electrode channel locations to 2D maps.

Figure 3. Attention module. Where ${\hat{\mathbf{U}}}_{sSE}$ extracts channel attention and ${\hat{\mathbf{U}}}_{cSE}$ extracts spectral attention.

Figure 4. DO-Conv process. The calculation method of figure (a) is called feature composition, and the calculation method of the figure (b) is called kernel composition.

Figure 5. Over-parameterised convolution module. The structure of this module for frequency and spatial feature learning.

Figure 6. The layout of GRU module for temporal feature acquisition.

Figure 7. Internal structure of GRU. The update gate and reset gate are incorporated.

Table

Algorithm 1: Emotion recognition based on AB-DPCGRU
Input: SEED, SEED-IV
Output: The predicted result of emotion recognition R
Preprocessing:
Use a bandpass filter to filter out five frequency bands to form 1-dimensional time series features.;
Calculate the frequency domain features of 5 frequency bands using differential entropy and obtain two-dimensional data according to Equations (2–4);
2D features of 5 frequency bands are superimposed to form 3D features.;
for max iters of AB-DPCGRU do
Calculation Attention:;
Calculate band and channel attention values through frequency and spatial attention;
Calculation Over-parameterized Convolution:;
Compression of multiple layers into a tight single layer representation;
Calculation Gated Recurrent Network Module:;
Calculate y according to Eqs. (5(5) $\begin{array}{rl}& z_t=\sigma (W_z\cdot [h_{t-1},x_t])\end{array}$(5) ), (6(6) $\begin{array}{rl}& r_t=\sigma (W_r\cdot [h_{t-1},x_t])\end{array}$(6) ), (7), (8), (9);
Calculate Out according to Eq. (10(10) $\text{ OUT }=A{y}_n=[\text{ out}{}_1,\text{ out}{}_2,\dots ,\text{ out}{}_M]$(10) );
Calculate the loss value according to Eq.(11);
Update AB-DPCGRU parameters;

Figure 8. The protocol used in the emotion experiment.

Table 1. Network parameter selection.

Name	Parameter
Learning rate	0.0001
Batch size	64
Epoch	200
Optimizer	Adamw
Activate function	Sigmod, Relu
Weight decay	0.001

Figure 9. The confusion graph of the SEED dataset.

Figure 10. Five-fold cross-validation accuracy.

Table 2. Ablation studies of each module.

	Accuracy	Kappa	F1-score
Enter raw data	0.6740 ± 0.1135	0.5429 ± 0.1762	0.7039 ± 0.1028
Eliminate the scSE module	0.9220 ± 0.0587	0.8845 ± 0.0633	0.9118 ± 0.0407
Overparameterized convolution replaced by standard convolution	0.8979 ± 0.0513	0.8625 ± 0.0798	0.8757 ± 0.0642
Eliminate the GRU module	0.8250 ± 0.0329	0.8003 ± 0.0679	0.7928 ± 0.0584
AB-DPCGRU	0.9399 ± 0.0277	0.9059 ± 0.0610	0.9162 ± 0.0346

Table 3. Performance comparison of the models on the SEED dataset.

	Accuracy	Kappa	F1-score
HCNN	0.8563	0.7963	0.8039
DGCNN	0.9040	0.8575	0.8722
Bi-HDM	0.9115	0.8626	0.8845
RGNN	0.9224	0.8816	0.9075
AB-DPCGRU	0.9399	0.9059	0.9162

Table 4. Performance comparison of the models on the SEED-IV dataset.

	Accuracy	Kappa	F1-score
HCNN	0.6918	0.6079	0.6831
DGCNN	0.7245	0.6598	0.7276
Bi-HDM	0.7203	0.6320	0.7137
RGNN	0.7458	0.6619	0.7682
AB-DPCGRU	0.77.04	0.7046	0.7723