Less-supervised learning with knowledge distillation for sperm morphology analysis

Figures & data

Figure 1. An example of the images in MHSMA dataset. Both images represent the same sperm. One is 128x128 pixels, and the other one is cropped 64x64 pixels.

Figure 2. Images and labels of MHSMA dataset for different parts of sperm such as vacuole, tail, head, and acrosome.

Table 1. Number of positive and negative samples in MHSMA dataset. There are 1,540 sperm images in the dataset labelled as normal or abnormal.

Label	Positive	Negative
Acrosome	1,086	454
Head	1,122	418
Vacuole	1,301	239
Tail	1,471	69

Figure 3. The AD procedure in our proposed method. This flowchart shows how we determine whether a raw image is normal or abnormal.

Figure 4. Visualized summary of our method, an offline distillation approach. LSn is the $\mathit{n}\mathit{th}$ layer of the student network and LTn is the teacher one. Also, this figure roughly shows which layers we use as critical layers.

Figure 5. Distance vectors for normal and abnormal samples. This figure shows the distance of several samples of normal and abnormal sperm that we detect anomalies based on that. This graph is obtained from the actual data of this dataset after training the model.

Figure 6. The architecture of the VGG16 network encoder (without the linear layers). This network is used as a teacher network in our method.

Figure 7. Model A: a simple network proposed to analyze the distillation effect. This network is used as a student network in our method.

Figure 8. Model B: the architecture of proposed student network. This network is used as a student network in our method.

Figure 9. Example of a sperm image in the dataset with flip augmentation applied during the training phase.

Figure 10. Example of a sperm image in the dataset with rotate augmentation applied during the training phase.

Figure 11. Gradient vector calculated from an image. This vector is used to add an adversarial attack to the image based on the value of gradients.

Figure 12. The steps of creating an attacked image from a normal image. Summing up the image with the epsilon portion of the sign of the gradients creates an attacked image.

Table 2. The outcomes of each part of sperm produced by the proposed method. The optimal setup of the suggested approach on the test set yielded these results.

Name	ROC/AUC	Precision	Recall	$F_{0.5}$ Score
Head	70.4	77.7	90.9	80.0
Vacuole	87.6	96.0	81.7	92.7
Acrosome	71.1	78.5	78.9	78.6

Table 3. Confusion matrix of the proposed method for each part of the sperm. These results are achieved from the evaluation phase on the test set.

Label	Confusion Matrix
Head	199 TP	20 FN
	57 FP	24 TN
Vacuole	214 TP	48 FN
	9 FP	29 TN
Acrosome	168 TP	45 FN
	46 FP	41 TN

Table 4. Comparison of our results with those achieved by Ghasemian et al. (2015) and Javadi and Mirroshandel (2019) on each part of sperm evaluated on the test set of the MHSMA dataset.

Label	Method	ROC/AUC	Precision	Recall	$F_{0.5}$ Score	MCC
Head	Proposed Method	70.4	77.7	90.9	80.0	+ 0.2572
	Ghasemian et al.	47.3	76.7	71.8	75.7	+0.0511
	Javadi and Mirroshandel	77.8	83.5	85.4	83.9	+ 0.4053
Vacuole	Proposed Method	87.6	96.0	81.7	92.7	+ 0.4416
	Ghasemian et al.	64.0	83.2	93.6	85.1	+0.3492
	Javadi and Mirroshandel	88.1	94.4	95.8	94.7	+ 0.5910
Acrosome	Proposed Method	71.1	78.5	78.9	78.6	+ 0.2608
	Ghasemian et al.	-	-	-	-	-
	Javadi and Mirroshandel	83.9	85.9	80.3	84.7	+ 0.4618

Figure 13. Boxplot of total loss mean for two groups. This plot indicates that the mean of the two groups is different.

Table 5. ANOVA test results for comparison of our results with those achieved by Javadi and Mirroshandel (2019).

Name	Statistic	p-value
Head	5.29	0.02
Vacuole	5.55	0.02
Acrosome	1.47	0.23

Figure 14. Anomaly localization map on abnormal samples from the test set. This map shows which parts of the image the model paid more attention to.

Table 6. Results achieved with distinct $\mathit{\lambda }$ values on validation set. Different $\mathit{\lambda }$ values can balance the effect of multiple losses.

Lambda ($\mathit{\lambda }$)	Label	Threshold	ROC/AUC	Precision	Recall	$F_{0.5}$ Score
0.1	Head	1.94	69.9	79.2	88.6	80.9
	Vacuole	2.10	64.1	86.9	92.3	88.0
	Acrosome	1.98	60.3	76.2	88.5	78.4
0.01	Head	0.77	70.5	80.6	77.8	80.0
	Vacuole	0.70	81.5	88.7	97.6	90.3
	Acrosome	0.67	68.0	76.6	92.0	79.2
0.001	Head	0.40	68.7	79.2	80.1	79.4
	Vacuole	0.53	82.5	89.1	97.6	90.7
	Acrosome	0.50	65.4	75.0	94.8	78.3

Figure 15. The result of using different architectures for student networks. Model a has a simple architecture. Model B has the proposed student architecture. Model C has the same architecture as the teacher network.

Table 7. Number of trainable parameters in three proposed models to show the effect of distillation. Changing the architecture of each network changes the number of its trainable parameters.

Layers	Number of trainable parameters
Model A	334,640
Model B	1,496,080
Model C	14,715,968

Table 8. Effect of critical layers on the performance of the proposed method. These results show how changing the number of critical layers in loss calculation can affect performance.

Set	Layers	Label	Threshold	ROC/AUC	Precision	Recall	$F_{0.5}$ Score
Validation Set	Just Output	Acrosome	0.21	64.8	76.8	85.6	78.4
		Head	0.16	65.3	77.4	97.2	80.7
		Vacuole	0.25	77.0	88.0	98.1	89.8
	2 Last Critical Layer	Acrosome	0.30	65.8	74.3	94.8	77.7
		Head	0.30	69.7	75.7	95.5	78.9
		Vacuole	0.36	80.8	90.8	94.7	91.6
	4 Last Critical Layer	Acrosome	0.67	68.0	76.6	92.0	79.2
		Head	0.77	70.5	80.6	77.8	80.0
		Vacuole	0.70	81.5	88.7	97.6	90.3
Test Set	Just Output	Acrosome	0.21	67.7	75.5	85.4	77.3
		Head	0.16	67.8	77.3	95.0	80.3
		Vacuole	0.25	78.8	88.7	98.5	90.5
	2 Last Critical Layer	Acrosome	0.30	69.7	73.4	95.8	77.0
		Head	0.30	66.5	76.3	95.4	79.5
		Vacuole	0.36	80.0	90.4	93.1	90.9
	4 Last Critical Layer	Acrosome	0.67	71.1	78.5	78.9	78.6
		Head	0.77	70.4	77.7	90.9	80.0
		Vacuole	0.70	87.6	96.0	81.7	92.7

Table 9. Model performance metrics comparison with and without data augmentation and adversarial attack.

Set	Mode	Label	Threshold	ROC/AUC	Precision	Recall	$F_{0.5}$ Score
Validation Set	Without augmentation and adversarial attack	Acrosome	0.58	61.4	75.0	96.6	78.5
		Head	0.51	66.1	77.7	90.9	80.0
		Vacuole	0.40	77.6	90.1	87.6	89.6
	With augmentation and without adversarial attack	Acrosome	0.49	63.3	77.1	90.8	79.5
		Head	0.42	67.6	80.3	79.0	80.1
		Vacuole	0.32	78.0	89.8	88.5	89.5
	With augmentation and adversarial attack	Acrosome	0.67	68.0	76.6	92.0	79.2
		Head	0.77	70.5	80.6	77.8	80.0
		Vacuole	0.70	81.5	88.7	97.6	90.3
Test Set	Without augmentation and adversarial attack	Acrosome	0.58	63.7	72.7	96.2	76.4
		Head	0.51	63.4	76.6	92.7	79.4
		Vacuole	0.40	76.6	91.5	91.3	91.4
	With augmentation and without adversarial attack	Acrosome	0.49	64.8	73.9	91.5	76.8
		Head	0.42	64.7	77.8	78.5	78.0
		Vacuole	0.32	78.1	91.6	92.0	91.7
	With augmentation and adversarial attack	Acrosome	0.67	71.1	78.5	78.9	78.6
		Head	0.77	70.4	77.7	90.9	80.0
		Vacuole	0.70	87.6	96.0	81.7	92.7

Appendix A. Table of abbreviations used in the text with their description.

Abbreviation	Description
SMA	Sperm Morphology Analysis
DL	Deep Learning
KD	Knowledge Distillation
ML	Machine Learning
MHSMA	Modified Human Sperm Morphology Analysis
AD	Anomaly Detection
IVF	In Vitro Fertilization
HuSHeM	Human Sperm Head Morphology
ICSI	Intracytoplasmic Sperm Injection
PCA	Principal Component Analysis
SIFT	Scale-invariant Feature Transform
KNN	K-Nearest Neighbor
LDA	Linear Discriminant Analysis
WHO	World Health Organization
HDC	Hybrid Dilated Convolution
SVM	Support Vector Machine
HSMA-DS	Human Sperm Morphology Analysis Dataset
GeNAS	Genetic Neural Architecture Search
SMIDS	Sperm Morphology Image Data Set
DMTL	Deep Multi-Task Learning
CNN	Convolutional Neural Network
CCD	Charge-Coupled Device
FGSM	Fast Gradient Sign Method
MNIST	Modified National Institute of Standards and Technology dataset
AUC	Area Under Curve
ROC	Receiver Operating Characteristic
PGD	Projected Gradient Descent
GBP	Guided Backpropagation
MCC	Matthews Correlation Coefficient
ReLU	Rectified Linear Unit
TP	True Positive
FP	False Positive
TN	True Negative
FN	False Negative
TPR	True Positive Rate
FPR	False Positive Rate
CS	Cosine Similarity
MSE	Mean Squared Error
SCIAN-MorphoSpermGS	gold-standard for morphological sperm analysis
VGG	Visual Geometry Group
GAN	Generative Adversarial Network

Appendix A describes various abbreviations and acronyms used throughout the paper.

Ghasemian F, Mirroshandel SA, Monji-Azad S, Azarnia M, Zahiri Z. 2015. An efficient method for automatic morphological abnormality detection from human sperm images. Comput Methods Programs Biomed. 122(3):409–420. doi: 10.1016/j.cmpb.2015.08.013.

 PubMed Web of Science ®Google Scholar

Javadi S, Mirroshandel SA. 2019. A novel deep learning method for automatic assessment of human sperm images. Comput Biol Med. 109:182–194. doi: 10.1016/j.compbiomed.2019.04.030.

 PubMed Web of Science ®Google Scholar