Joint Dual-Structural Constrained and Non-negative Analysis Representation Learning for Pattern Classification

Figures & data

Table 1. Comparative characteristics of the related works on discriminative ADL models. The regularization term of the DSNAR model corresponding to each technical component is shown in parentheses.

ADL Models	Class-Oriented	Label-Embedded Dictionary	Label-Embedded Representation	Nonnegative Representation
ADL-SVM	✘	✘	✘	✘
DADL	✘	✓	✘	✘
CADL	✓	✘	✓	✘
SK-ADL	✘	✘	✓	✘
SADL	✘	✓	✓	✘
SDADL	✓	✓	✓	✘
SADL-DFP	✘	✓	✓	✘
DSNAR	✓ (${∥X\odot S∥}_F^2$)	✓ ($\parallel L-AX{\parallel }_F^2$)	✓ ($\parallel H-AX{\parallel }_F^2$)	✓ ($X\ge 0$)

Figure 1. Framework of the proposed DSNAR model.

Table

Algorithm 1 Algorithm for solving our proposed model (4)
Input: Training data $Y$, latent space matrix $L$, model parameter $\alpha$,$\beta$, sparsity level $T_0$ and convergence tolerance $\epsilon =1e-6$, maximum iteration $T_{max}$.
Output: The analysis dictionary $\mathrm{\Omega }$ and transformation matrix $A$.
1: Initialize $\mathrm{\Omega }$ by (22), $A$ by (23), $Z=X=L$, $C_1={\mathbf{0}}^{k\times n}$, $\rho =1.01$, $\mu ={10}^{-8}$, ${\mu }_{max}={10}^8$, $t=1$.
2: while not converge and $t<T_{max}$ do
3: fixing $\mathrm{\Omega },Z$, update $A,X$ by solving model (13) using K-SVD method;
4: fixing $\mathrm{\Omega },A,X$, update $Z$ by Eq. (19)(19) $Z=(2\beta Q_1+\mu X+C_1)/(2\beta +\mu )s.t.Z\ge 0$(19) ;
5: fixing $A,X,Z$, update $\mathrm{\Omega }$ by Eq. (22)(22) $\mathrm{\Omega }=X{Y}^T{\left(Y{Y}^T+\gamma I\right)}^{-1}$(22) ;
6: update $C_1=C_1+\mu (Z-X)$, $\mu =min(\rho \mu ,{\mu }_{max})$
7: check the convergence condition: ${∥Z^t-X^t∥}_{\mathrm{\infty }}<\epsilon$, and $t=t+1$
8: end while

Figure 2. Convergence curves of the DSNAR model on the EYaleB and Scene15 datasets.

Table 2. The statistical information of the benchmark datasets and features.

Datasets	Feature/dimension	Instances(total/use)	Class(total/use)	Train/test
EYaleB	Random face/504	64/64	38/38	20/44
AR	Random face/540	26/26	126/100	10/16
LFW	Cropped face/1024	11–20/all	86/86	8/rest
Scene15	SIFT/3,000	200–400/all	15/15	100/rest
UCF50	SIFT/3,000	31–800/all	102/102	30/rest

Table 3. Classification accuracy (%) comparison on different datasets. The best results are in bold.

DL Models	EYaleB	AR	LFW	Scene15	UCF50
SRC	96.5	97.5	38.1	91.8	68.4
D-KSVD	94.1	88.8	35.9	89.1	57.8
LC-KSVD	96.7	97.8	36.7	92.9	70.1
ADL+SVM	95.4	96.1	32.6	90.1	72.3
SK-ADL	96.7	97.7	38.7	97.4	74.6
SLC-ADL	97.0	97.2	38.9	97.5	78.1
SDADL	97.0	97.8	40.1	98.1	77.8
DSNAR	97.3	98.1	40.4	98.7	78.5

Table 4. The average time (ms) for training procedure of different models.

DL Models	EYaleB	AR	LFW	Scene15	UCF50
D-KSVD	48.9	39.2	43.9	61.3	45.8
LC-KSVD	76.5	52.5	85.0	93.1	91.6
SK-ADL	38.4	40.0	48.3	44.4	43.3
SDADL	62.2	65.6	99.5	65.5	74.4
DSNAR	35.5	32.9	44.5	37.1	43.2

Table 5. The average time (ms) for classification procedure of different models.

DL Models	EYaleB	AR	LFW	Scene15	UCF50
SRC	39.93	41.24	38.12	70.66	92.37
LC-KSVD	0.426	0.442	0.351	0.675	0.794
SK-ADL	0.029	0.078	0.052	0.113	0.169
SDADL	0.023	0.063	0.051	0.100	0.158
DSNAR	0.024	0.063	0.050	0.105	0.157

Figure 3. Confusion matrix of the ground truth on Scene 15 dataset.

Figure 4. Results of parameter selection of $\alpha$ and $\beta$ on AR and UCF50 datasets.

Table 6. Best parameter settings for DSNAR model by cross validation.

Parameters	EYaleB	AR	LFW	Scene15	UCF50
$\alpha$	50	60	4	20	20
$\beta$	12	20	1	14	11

Table 7. Classification accuracy (%) with different numbers of atoms on the LFW dataset.

Atoms#	LC-KSVD	ADL+SVM	SK-ADL	SLC-ADL	DSNAR
86	29.3	27.9	29.6	31.4	32.3
172	34.1	29.7	34.3	34.6	35.7
258	34.8	30.3	36.1	36.8	37.5
344	35.4	31.1	37.0	37.9	38.9
430	36.9	31.7	38.2	38.6	39.9
516	36.7	32.6	38.7	38.9	40.4

Data availability statement

All data generated or analyzed during this study are included in this published article.