RePoint-Net detection and 3DSqU² Net segmentation for automatic identification of pulmonary nodules in computed tomography images

Figures & data

Figure 1. Proposed framework of nodule identification. Graphical illustration of the proposed pipeline that contains three modules. RePoint-Net was trained with dataset LIDC/IDRI the trained model as nodule or non-nodule. The $3\mathit{DSq}{U}^2\mathit{Net}$ model segmented of lung and nodules of detected in RePoint-Net. classification of nodules candidate by proposed 3DCNN as nodule benign or nodule malignant.

Figure 2. Proposed structure for RPN anchor selection.

Figure 3. Schematic representation of the RePoint-Net method for identifying nodules pulmonary.

Table 1. Variables used in our method.

Variable	Comment
$P_K$	A set with NK Rep-Points
$K_{\mathit{rep}}$	The number of channels of the label tensor induced by Rep-Points
$K_{\mathit{rpn}}$	The number of channels of the label tensor induced by Point-Net
$\mathit{w}$	The number of columns of the last shared convolutional tensor
$\mathit{h}$	The number of rows of the last shared convolutional tensor
$T_{\mathit{Point}-\mathit{Ne}{\mathit{t}}_{\mathit{po}}}$	The IoU threshold for positive anchors
$T_{\mathit{Point}-\mathit{Ne}{\mathit{t}}_{\mathit{no}}}$	The IoU threshold for negative anchors
$T_{\mathit{Point}-\mathit{Ne}{\mathit{t}}_{\mathit{bs}}}$	The threshold for the number of anchors for Point-Net training
$r_{\mathit{Point}-\mathit{Ne}{\mathit{t}}_{\mathit{fg}}}$	The ratio of the anchors that are labelled as foreground

Figure 4. The architecture of the encoder network.

Figure 5. The architecture of the $3\mathrm{DSq}{\mathrm{U}}^2$ Net network for segmentation of pulmonary nodules.

Figure 6. Our proposed 3D deep CNN architecture.

Table 2. Annotation of nodule characteristic scoring.

Attribute	Significance	Rating	(range),threshold
Margin	How well defined the margin is	Poorly defined, near poorly defined, medium margin, near sharp, sharp	(1–5), 3
Spiculation	How spiculated the nodule is	No spiculation, nearly no spiculation medium spiculation, near marked spiculation marked spiculation	(1–5), 3
Calcification	Pattern of calcification	Popcorn, laminated, solid, non-central central, absent	(1–6), 3
Sphericity	Shape of nodule	Linear, ovoid/linear, ovoid, ovoid/round round	(1–5), 3
Lobulation	How irregular the margin is	No lobulation, nearly no lobulation, medium lobulation, near marked lobulation marked lobulation	(1–5), 3
Internal structure	Internal composition of nodule	Soft tissue, fluid, fat, air	(1–4), 3
Subtlety	Difference between outside tissue and nodule	Extremely subtle, moderately subtle, fairly subtle, Moderately obvious	(1–5), 3
Texture	Nodule’s texture	Non-solid, non-solid, partially solid, solid, solid	(1–5), 3

Table 3. Definition of TP, FP, FN, TN.

Category	Actual lesion	Actual non-lesion
Predicted Lesion	True Position (TP)	False Position (FP)
Predicted Non-Lesion	False Negative (FN)	True Negative (TN)

Table 4. System performance of nodule candidate detection.

Method	Total Number of Candidates	Total number of Non-Nodules	Total Number of Nodules	Sensitivity (%)	Specificity (%)	DSC (%)	F1-score (%)
Total	2121	1111	1010
REPOINT-NET		1081	986	97.63	97.30%	77.83%	93.61

Table 5. Detection of nodules based on diameters with RePoint-Net.

Nodule size	Number of Nodules	Detect of nodules	TP	FP	Sensitivity (%)	Accuracy (%)	Specificity (%)
3–5 mm	237	217	205	12	91.55	90.72	93.49
5–8 mm	288	278	268	10	96.53	94.44	95.42
8–10 mm	132	130	125	5	98.48	95.45	96.23
10–20 mm	246	246	246	0	100	100	100
>20 mm	115	115	115	0	100	100	100
Total	1018	986	959	27	97.11	96.22	97.03

Figure 7. Likelihood ratios for positive and negative results of RePoint-Net.

Table 6. The segmentation results of the proposed model for 1018 pulmonary nodules category of texture.

Category of Texture	No. of Nodules	Detected Nodules	TP	FP	Sensitivity (%)	Accuracy (%)	Specificity (%)	DSC (%)	F1-score (%)
Solid	539	530	522	8	0.932	0.985	.985	0.968	0.957
Subsolid	293	287	280	7	0.892	0.976	.976	0.966	0.966
Non-Solid	186	180	165	15	0.892	0.919	.917	0.904	0.904
Total	1018	997	967	30	0.908	0.96	.959	0.946	0.942

Figure 8. The table label describes the ROC curve, which demonstrates the balance between sensitivity and specificity at different classification thresholds. An ideal classifier would achieve a TPR of 1.0 and an FPR of 0.0 at all threshold values, resulting in a ROC curve with a TPR of at least 0.93 at an FPR of 0.01 or below. The classifier’s performance is summarized by the area under the curve (AUC) with a range of 0.0 to 1.0, where 0.5 indicates random guessing and 1.0 shows a perfect classifier. The classifier discussed has an AUC of 0.986, indicating strong discrimination power between nodule-benign and nodule-malignant cases.

Table 7. ROC curve threshold table.

Threshold Values	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
FPR (1 -Specificity)	0.038	0.038	0.103	0.103	0.131	0.036	0.042	0.024	0.02	0.012	1
TPR (Sensitivity)	1	0.998	0.994	0.992	0.988	0.972	0.968	0.96	0.944	0.932	0

Table 8. The confusion matrix of our method.

Predicted	Actual
	Nodule-Malignant	Nodule-Benign
Nodule-Malignant	0.962	0.038
Nodule-Benign	0.053	0.947

Table 9. Classification of nodules using 3DCNN.

Type of Nodule	Number of Nodules	Detect of Nodules	True Positive	Specificity	FPR (1-Specificity)	TPR (Sensitivity)
Nodule-Benign	138	135	130	.964	.036	.942
Nodule-Malignant	880	865	855	.989	.011	.972

Table 10. IoU scores between different variants of U-Net and the proposed method.

Network Structure	IoU
Deep-Lung Zhu et al. (2018b)	46.03%
Traditional U-Net	49.25%
RUN: Residual U-Net Lan et al. (2018) Mobilenet-V2 Howard et al. (2017)	56.13% 72.73%
DeepLab-V3+ with X-65 encoder El-Bana et al. (2020)	79.1%
Xception-65 Chollet (2017)	78.77%
Our method	91.35%

Figure 9. Comparison of the IoU between our model and prior models. The results show that prior models outperform the baseline method implementation and standard.

Zhu W. 2018b. Deeplung: deep 3d dual path nets for automated pulmonary nodule detection and classification. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV); 12–15 March 2018; Lake Tahoe, NV, USA: IEEE. doi: 10.1109/WACV.2018.00079.

 Google Scholar

Lan T. 2018. Run: residual u-net for computer-aided detection of pulmonary nodules without candidate selection. arXiv preprint arXiv:1805.11856.

 Google Scholar

Howard AG. 2017. Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

 Google Scholar

El-Bana S, Al-Kabbany A, Sharkas M. 2020. A two-stage framework for automated malignant pulmonary nodule detection in CT scans. Diagnostics. 10(3):131. doi: 10.3390/diagnostics10030131.

 Web of Science ®Google Scholar

Chollet F. 2017. Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. p. 1251–1258. doi: 10.48550/arXiv.1610.02357.

 Google Scholar