Iterative Convolutional Encoder-Decoder Network with Multi-Scale Context Learning for Liver Segmentation

Figures & data

Figure 1. Examples of CT and MR abdominal organ images. (a) a CT image slice in axial view. (b) a T1-weighted sequence of MR image slice in axial view. (c) a T2-weighted sequence of MR image slice in axial view.

Figure 2. Network architecture.

Table

Algorithm 1. The procedure of iterative learning based on the proposed network
Input: $\left\{x_i,l_i,i=1\text{cdotsm}\right\}$ the training image pairs; $s_i^0$ the initial probability map with uniform distribution Output: the learned weights of the proposed networks (step $t=1,\dots ,K$)
repeat
Obtain the training set ${\mathrm{\Gamma }}^t=\left\{\left(x_i,s_i^{\left(t-1\right)}\right),l_i,i=1\text{cdotsm}\right\}$;
Train the proposed network described in Figure 2 using ${\mathrm{\Gamma }}^t$;
Calculate the posterior probability $s_i^t$ through Sigmoid activation function for $\left\{i=1\text{cdotsm}\right\}$;
Calculate the loss ${\mathrm{H}}_i^t$ via Equation (1)(1) $\mathrm{H}\left(L,S\right)=-\omega \cdot \frac{1}{N}\sum _{n=1}^N{l}_n\cdot log{s}_n+\left(1-\frac{2{\sum }_{n=1}^N{l}_n{s}_n}{{\sum }_{n=1}^N{l}_n+{\sum }_{n=1}^N{s}_n}\right),$(1) for $\left\{i=1\text{cdotsm}\right\}$;
Calculate $\mathrm{\Delta }\mathrm{H}=1\text{overm}\sum _{i=1}^m\left|{\mathrm{H}}_i^t-{\mathrm{H}}_i^{\left(t-1\right)}\right|$;
until $\mathrm{\Delta }\mathrm{H}<\epsilon$ ($\epsilon$ is a predefined threshold)

Figure 3. The training results of the 2D UNet++ and the proposed method over 80 epochs from (a) the 3Dircadb1 dataset, (b) the Sliver07 dataset, and (c) the CHAOS19 dataset.

Figure 4. The boxplot of DICE results using four methods on CT images of three datasets. A star indicates a statistical significant difference of the median DICE compared to the previous column.

Figure 5. Typical liver segmentation results on CT images of the 3Dircadb1 (top row), the Sliver07 (middle row), and the CHAOS19 (bottom row) datasets by using four methods. (a) 2D U-Net. (b) 2D Auto-Net. (c) 2D UNet++. (d) the proposed method. Green contours indicate the ground truth segmentation, and red contours indicate the automatic segmentation by the algorithm.

Table 1. The mean and standard deviation of quantitative measures for liver segmentation on three datasets.

	Methods	DICE	RAVD	ASSD(mm)	MSSD(mm)
3Dircadb1	2D U-Net	0.925±0.023	6.030±3.428	3.927±2.150	55.300±20.047
	2D Auto-Net	0.940±0.021	4.899±3.631	2.997±1.588	29.457±6.317
	2D UNet++	0.941±0.028	4.859±5.510	1.277±1.124	38.079±19.871
	the proposed method	0.955±0.012	3.782±2.306	1.071±0.981	32.939±12.252
Sliver07	2D U-Net	0.918±0.032	7.720±5.917	4.839±2.224	55.963±45.400
	2D Auto-Net	0.933±0.031	5.831±4.376	4.183±2.210	39.682±12.647
	2D UNet++	0.938±0.032	4.127±3.369	3.660±2.195	40.635±15.212
	the proposed method	0.954±0.014	3.166±2.721	2.059±0.748	29.448±9.285
CHAOS19	2D U-Net	0.931±0.016	6.064±3.460	3.832±1.811	45.633±26.153
	2D Auto-Net	0.947±0.016	4.698±2.437	2.493±1.066	28.485±7.028
	2D UNet++	0.952±0.021	3.025±2.231	1.217±0.434	36.245±11.945
	the proposed method	0.967±0.011	2.296±1.980	0.929±0.341	28.948±8.856

Figure 6. The training results of the 3D UNet++ and the proposed method over 160 epochs from (a) the T1-DUALin sequence, (b) the T1-DUALout sequence, and (c) the T2-SPIR sequence.

Figure 7. The boxplot of DICE results using four methods on multi-sequence MR images of the CHAOS19 dataset. A star indicates a statistical significant difference of the median DICE compared to the previous column.

Figure 8. Typical liver segmentation results on MR images with the T1-DUALin (top row), the T1-DUALout (middle row), and the T2-SPIR (bottom row) sequences by using four methods. (a) 3D U-Net. (b) 3D Auto-Net. (c) 3D UNet++. (d) the proposed method. Green contours indicate the ground truth segmentation, and red contours indicate the automatic segmentation by the algorithm.

Table 2. The mean and standard deviation of quantitative measures for liver segmentation on multi-sequence MR images of the CHAOS19 dataset.

	Methods	DICE	RAVD	ASSD(mm)	MSSD(mm)
T1-DUALin	3D U-Net	0.916±0.021	4.162±3.420	1.268±0.689	19.378±6.962
	3D Auto-Net	0.936±0.018	3.987±2.949	0.711±0.362	19.161±5.346
	3D UNet++	0.938±0.019	3.663±2.608	0.878±0.681	18.856±5.193
	the proposed method	0.951±0.012	3.299±1.603	0.834±0.625	16.750±4.328
T1-DUALout	3D U-Net	0.911±0.021	4.990±1.535	1.601±1.809	24.347±17.257
	3D Auto-Net	0.914±0.024	4.674±6.358	1.245±0.640	23.819±9.686
	3D UNet++	0.930±0.021	4.515±2.071	1.466±0.830	22.369±8.108
	the proposed method	0.942±0.012	3.040±1.621	1.102±1.101	20.461±5.989
T2-SPIR	3D U-Net	0.908±0.019	7.090±2.279	1.825±1.080	26.990±9.357
	3D Auto-Net	0.926±0.019	5.340±2.628	1.285±0.805	25.304±9.741
	3D UNet++	0.918±0.014	5.891±2.164	1.782±1.146	23.089±8.905
	the proposed method	0.935±0.018	3.654±2.651	1.397±0.639	22.240±10.163

Table 3. The overall quantitative results of liver segmentation on CT and MR images as mean by using the Transformer-based methods and the proposed method.

	Methods	DICE	RAVD	ASSD(mm)	MSSD(mm)
CT	Swin-Unet	0.947	4.103	2.152	28.541
	the proposed method	0.958	3.081	1.353	30.445
MR	UNETR	0.950	3.684	1.276	18.722
	the proposed method	0.943	3.331	1.111	19.817

Table 4. The overall quantitative results of liver segmentation on MR images as mean by using the proposed method with different scales.

Methods	DICE	RAVD	ASSD(mm)	MSSD(mm)
IterConv-SSC	0.933	4.035	1.326	20.563
IterConv-MSC	0.943	3.331	1.111	19.817

Figure 9. The DICE results of liver segmentation on all CT images as mean by using four steps of the proposed method.

Table 5. The model complexity of different networks.

Methods	Params	FLOPs
2D U-Net	1.95M	15.98G
2D Auto-Net	1.95M	16.02G
2D UNet++	2.29M	34.57G
the proposed method (2D)	2.32M	36.73G
3D U-Net	4.12M	167.58G
3D Auto-Net	4.12M	168.18G
3D UNet++	6.87M	664.63G
the proposed method (3D)	6.96M	697.08G