Liver segmentation on a variety of computed tomography (CT) images based on convolutional neural networks combined with connected components
VNU Journal of Science: Comp. Science & Com. Eng, Vol. 36, No. 1 (2020) 25-37
Original Article
Liver Segmentation on a Variety of
Computed Tomography (CT) Images Based on Convolutional
Neural Networks Combined with Connected Components
Hoang Hong Son1, Pham Cam Phuong2,
Theo van Walsum3, Luu Manh Ha1,3,*
1VNU University of Engineering and Technology, Vietnam National University, Hanoi,
144 Xuan Thuy, Cau Giay, Hanoi, Vietnam
2The Nuclear Medicine and Oncology center, Bach Mai hospital,
78 Giai Phong, Phuong Dinh, Dong Da, Hanoi, Vietnam
3BIGR, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
Received 17 December 2019
Revised 23 January 2020; Accepted 23 March 2020
Abstract: Liver segmentation is relevant for several clinical applications. Automatic liver segmentation
using convolutional neural networks (CNNs) has been recently investigated. In this paper, we propose a
new approach of combining a largest connected component (LCC) algorithm, as a post-processing step,
with CNN approaches to improve liver segmentation accuracy. Specifically, in this study, the algorithm
is combined with three well-known CNNs for liver segmentation: FCN-CRF, DRIU and V-net. We
perform the experiment on a variety of liver CT images, ranging from non-contrast enhanced CT images
to low-dose contrast enhanced CT images. The methods are evaluated using Dice score, Haudorff
distance, mean surface distance, and false positive rate between the liver segmentation and the ground
truth. The quantitative results demonstrate that the LCC algorithm statistically significantly improves
results of the liver segmentation on non-contrast enhanced and low-dose images for all three CNNs. The
combination with V-net shows the best performance in Dice score (higher than 90%), while the DRIU
network achieves the smallest computation time (2 to 6 seconds) for a single segmentation on average.
The source code of this study is publicly available at https://github.com/kennyha85/Liver-segmentation.
Keywords: Liver segmentations, CNNs, Connected Components, Post processing.
1. Introduction*
patient without treatment is less than 15% [2].
Liver cancer is more common in sub-Saharan
Africa and Southeast Asia regions compared
with Europe and United States. In some
developing countries such as Vietnam, liver
cancer is the most common type of cancer [3, 4].
Liver cancer has one of the highest mortality
rates for cancers worldwide [1], with a total of
approximately 800,000 new cases annually. In
general, the 5-year survival rate of liver cancer
_______
* Corresponding author.
E-mail address: [email protected]n
25
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
26
Liver radiofrequency ablation (RFA) has
minimally invasive procedure without producing
toxic side-effects such as radioembolization and
chemoembolization [5, 6]. Furthermore, the liver
of patients treated with RFA recovers in only a
few days after receiving the intervention [7].
become a popular treatment for liver cancer due
to its several advantages. This type of treatment
is appropriate in the early stage or in cases of
multiple tumors. RFA is a relatively low-risk
L
Figure 1. A typical contrast enhanced CT image of the liver (A) and the 3D segmentations of the liver,
vessels and tumors (B). The volume rendering provides 3D visualization
of the liver and the tumor in a RFA planning stage.
The CT imaging modality is often used for
diagnosing liver cancer and planning the RFA
treatment procedure for liver cancer. The 3D
liver segmentation on the CT images of the liver
is thus relevant for RFA treatment of liver
cancer. In the planning stage, the liver
segmentation acts as a region of interest, which
contains the liver tumor and the liver vessels (see
Figure 1). First, the visualization of the 3D liver
segmentation provides adequate information to
enable the radiologist to decide on the process of
ablator insertion such that the trajectory of the
insertion does not reach the critical parts such as
bones, vessels and the kidneys. Second, the liver
segmentation may also act as a mask region for
liver registration using pre-operative, intra-
operative and post-operative CT images of the
RFA liver intervention [8, 9]. Typically, the liver
segmentation can be performed manually by a
radiologist as a slice-by-slice approach. Because
this manual approach requires tedious work and
a substantial amount of time, it does not match
the clinical workflow well. Therefore, liver
segmentation using computer-based automatic
and semiautomatic strategies has recently
become an active research field. However, the
noise due to lowering radiation dose, the low
contrast between the liver and nearby organs,
liver movement due to breathing motion, and the
differences in size, shape and voxel intensity
inside the liver across different patients present
as current challenges to the implementation of
3D liver segmentation in the clinical setting.
Several liver segmentation methods have been
proposed in the literature and have high potential
to be applied in clinical practice. In general,
those methods can be classified into two main
groups. The first group contains classical
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
27
statistical and image-processing approaches
such as region growing, active contour,
deformable models, graph-cuts, statistical shape
model [10, 11]. These methods use hand-crafted
features, and thus provide limited feature
representation capability. The second group
consists of Convolutional Neural Networks
(CNNs), which have achieved remarkable
success in many fields in the medical imaging
domain such as object classification, object
detection, and anatomical segmentation. Several
I
CNN approaches have shown improved
accuracy performance and are comparable to
manual annotations by experts in oncology and
radiology [12]. This success can be attributed to
the ability of CNNs to learn a hierarchical
representation of spatial information of CT
images [13]. CNN approaches, how require large
amount of data to train the models which is one
of the main limitations in medical imaging
research domain because medical image sharing
is often limited due to privacy concerns.
Figure 2. Illustration of 2D U-net architecture for liver segmentation using CT images with the inputs as a 2D
image and the output as a predicted map of the liver. The networks contain four levels of the hierarchical
representation. The skip connections provide linear combinations of the feature maps at the same level of up
sampling and down sampling paths.
In current liver segmentation, CNN-based
segmentation algorithms have considerably
outperformed the classical statistical/image-
processing-based approaches [12, 14-16]. U-net,
one of the most well-known CNN architectures,
introduced by Ronneberger et al. (2015), has
received high rankings in several competitions in
the field of medical image segmentation [12],
and Christ et al. (2016) have successfully
segmented the liver using a U-net architecture
[15] (see Figure 2). Christ et al. (2017) further
developed a fully convolutional neural network
(FCN) based on the U-net architecture to
segment the liver in both CT and MRI images,
achieving a mean of Dice score of 94% with
fewer than 100 training images [14]. Lu et al.
(2015) have proposed a 3D CNN-GC method
that combines a 3D fully convoluted neural
network and graph cuts to achieve automatic
liver segmentation in CT images with an
accuracy of VOE of 9.4% on average [7]. Li et
al. (2018) have also introduced the
H-dense U-net for automatic liver segmentation,
coupling intra-slice information using 2D dense
U-net and inter-slice information using a 3D
counterpart, and obtained the mean of DICE of
96.1% [17]. Bellver et al. (2017) have further
improvised the original OVOS neural network,
called DRIU, to segment the liver in CT images
and achieved comparative results [18]. The
number of publications relating to liver
segmentation using a CNN has been increasing
dramatically and most of them participate in the
MICCAI grand challenge for liver segmentation
(LiTS). Those CNNs, in general, can be classified
into two categories: 2D Fully Convolutional
Networks (2D FCNs) [14, 15, 18] and 3D Fully
Convolutional Networks (3D FCNs) [13, 17, 23].
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
28
While 3D CNNs require greater computational
conclusion section summarizes the findings in
this study.
complexity and consume more VRAM memory,
the segmentation performance of 3D FCN versus
2D FCN still remains under debate [16].
As a machine learning classification family,
CNNs perform convolutional filter image
classification to segment the objects and as a
result may contain several mis-classified voxels.
Therefore, post-processing techniques may be
applied to improve liver segmentation using
CNNs. Conditional Random Forest (CRF) is a
well-known method for post-processing of liver
segmentation, but based on our previous study
[19], CRF does not work well with CNN-based
liver segmentation of low-dose/non-contrast CT
images. Milletari et al. (2016) further states that
“post-processing approaches such as connected
components analysis normally yield no
improvement” [13]. Considering the paucity of
studies, it is necessary to elucidate how post-
processing impacts the liver segmentation on
CT images.
Given that the liver is the largest organ in the
abdominal cavity, we hypothesize that the liver
segmentation should be the largest connected
component in the segmentations obtained from the
CNNs. The main contribution of our study is that we
propose a largest connected component LCC)
algorithm to improve the liver segmentation in CT
images using CNNs. To do this, we perform a full
search for the largest connected component based
on the connected component algorithm [20], and
then we apply the algorithm on the liver
segmentations generated by three well-known
CNN architectures: U-net + CRF [14], DRIU [18]
and V-net [13]. We evaluate the methods on three
datasets: Contrast enhanced CT images, low-dose
contrast enhanced CT image and low-dose,
non-contrast enhanced CT image.
2. Method
2.1. Convolution Neural network architectures
● Fully Convolutional Network (FCN)
combined with conditional random fields (CRF)
The Fully Convolutional Network (FCN)
combined with conditional random fields (CRF),
proposed by Christ et al. (2017), contains two 2D
U-net networks in a cascaded structure to
sequentially segment both the liver and liver
tumors [15]. U-net architecture is a well-known
FCN that is able to learn a hierarchical
representation of the image in the training stage
In this study, we re-implement the first
U-net network for the task of liver segmentation
using CT images. The U-net architecture
contains 19 layers in 4 levels and is divided into
two parts: The encoder (also called “contracting
path”) and the decoder (also called “expanding
path”). The encoder classifies the contextual
information of all of the pixels in the input image
via a process of hierarchical extractions, while
the decoder provides the spatial information of
the classified pixels to their corresponding
location in the original image. Furthermore, the
U-net skips several connections at different
levels to provide information of the feature maps
from the encoder section to the decoder section
at the same levels. Embedding the skipped
connections allows compensation of information
about the objects that can be lost after each layer
in the main path of U-net architecture.
The next sections are organized as follows: the
methods section briefly describes the three CNNs
architectures and LCC method; next, the
experiments section presents in detail the
implementation of the CNNs architectures, the data
used in the study and the criteria to evaluate the
performance of the proposed method. The results
are illustrated in section 4, which is followed by a
discussion of the results in section 1) The
The U-net input is 2D images and the output
is a 2D probability map as the result of a
soft prediction classifier for each pixel in the
original images.
For the optimization process, weighted
binary cross entropy CE is used as the objective
loss function:
퐶퐸 = − 푁1
(1)
푁
푖
∑
( )
푤푖푡푖 log 푠푖 ,
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
29
● DRIU: Deep retinal image understanding
where N is the number of pixels involved in the
training stage; t is the ground truth value, which
is either 0 or 1 when the pixel i is either
background or foreground; S is the soft
prediction score at the location pixel; i and w are
the weights defining the degree of importance of
the liver pixels. w is chosen as 1 over the
foreground region size.
Subsequently,
random field (CRF) is applied on the 2D
probability maps, enabling the combination of
both 3D spatial coherence and 2D appearance
information from the slice-wise U-net
segmentation [15].
i
DRIU was introduced by Bellver et al.
(2017) to segment the liver in abdominal contrast
enhanced CT images [18]. The network
architecture utilizes VGG-16 as the back-bone
network, removing the last classification layers,
i.e. the fully-connected layers, while maintaining
other layers such as the fully convolutional
layers, ReLU active function, and max-pooling
layers. Similar to U-net, the DRIU architecture
includes a contracting part and an expanding part
containing several paired convolutional layers
with the same size of feature map. The main
difference from U-net is that the feature map at
each level of the expanding part is achieved by
up-sampling the feature map in the lower layer
from the contracting part. In addition, in the
expanding path, the output of DRIU is a
combination of all feature maps at multiple scales
by rescaling them to the original image size and
then integrating them up into a single image. Thus,
the segmentation contains information of the liver
as a multiscale representation of the image. We
also use weighted Binary Cross Entropy loss
function for the optimization process.
i
i
i
a
3D-dense conditional
● V-Net: Fully CNNs for volumetric medical
image segmentation
While most CNNs utilize 2D convolution
kernels to segment objects in 2D images, the
V-net segments a 3D liver volume using 3D
convolution kernels embedded in a fully
convolutional neural network [13, 17]. The
V-net is more or less a 3D version of U-net and
also contains two parts: the down-sampling path
and the up-sampling path. The down-sampling
path compresses the original 3D images into
feature maps, while the up-sampling path
extracts the feature maps until the final output
reaches the original size of the input 3D image.
Similar to U-net, the skipped connections from
the encoding to the decoding path at the same
deep levels to provide spatial information of
each layer and thus further improve the accuracy
of the final segmentation prediction.
2.2. Largest connected component (LCC)
In order to remove isolated regions of false
segmentations of the liver generated by the
CNNs, we propose to apply a connected
component algorithm in the post-processing
stage. We first apply a 3D connected
component-labeling algorithm [20] and then
perform a full searching for the largest connected
component. Note that there should be a few
connected components with the liver
segmentation component as the largest one,
given that the liver is the largest organ in the
abdominal cavity. In the case that the largest
component is not the liver, the neural network
would not perform well and the segmentation
should be treated as a failed case.
In this study, we utilize Dice loss as the
objective function in the optimization process as
suggested in the original work [13]:
푁
푖
∑
2
푝 푔
푖
푖
퐷 =
,
푖
(2)
푁
푖
2
푁
푖
2
∑
∑
푝 +
푔
푖
where and are voxel values, either being 1 or 0,
of the predicted liver segmentation and the
ground truth, respectively, and N is the number
of voxels of the two images in the same size.
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
30
Table 1. The pseudocode of the largest connected
component algorithm
Segmentation (LiTS) challenge in the MICCAI
grand challenge [21]. The images were acquired
on a variety of CT scanners and protocols from
multiple medical centers. We used LiTS dataset
for training the three CNN models, like as
previous done in Bellver et al. (2017) [18].
The second dataset consists of 10 CT images
from the Mayo Clinic (Mayo), which were
acquired by a Siemens CT scanner under a
typical scanning protocol. The images are
contrast enhanced portal-venous phase, and
include several primary liver tumors. In order to
reduce the redundant slices, the images were
manually cropped in the z dimension such that
the liver region is preserved.
Algorithm LCC(segmentation)
labels = list of connected component of segmentation
LCC_label = 0
Largest_CC_size = 0
for label in labels:
if volume of label is larger than largest_CC_size
largest_CC_label = label
largest_CC_size = volume of label
Largest_LCC_segmentation = segmentation labeled
by LCC_label
return Largest_LCC_segmentation
The third and the fourth dataset are 15
contrast enhanced (EMC-LD) and 15 non-
contrast enhanced CT images (EMC-NC-LD),
respectively, which were randomly selected
from Erasmus MC PACS in 2014 [8]. The
images were acquired during radio frequency
ablation intervention under low-dose protocol,
resulting in noisy images due to the low radiation
dose (see Figure 4).
The datasets from Erasmus MC and Mayo
were manually annotated by two experts for
ground truth, which is used in the evaluation
section in this study, while the dataset from LiTS
challenge already is publicly available with the
liver segmentation ground truth segmented by
several experts.
3. Data and experiment setup
3.1. Clinical data
In this study, we perform experiments using
four datasets of CT images as in our previous
study [19], which contains several variants of
liver CT images: contrast enhanced, low-dose
contrast enhanced, and low-dose non-contrast
enhanced CT images. All of the confidential
information in the datasets were anonymized by
their own medical centers before taking part in
this study. The parameters of the datasets are
summarized in the Table 2.
The first dataset contains 115 contrast
enhanced CT images from the Liver Tumour
Table 2. Parameters of the datasets in the study
Dataset
Number of Resolution Spacing Number of Voltage
data
115
10
(mm)
(mm)
slices
(kVP)
-
100
80 - 120
80 - 120
LiTS
Mayo
EMC_LD
0.55 - 1.0 0.45 - 6.0
0.64 - 0.84 3.0
0.56 - 0.89 2 - 5
0.56 - 0.89 5
74 - 986
46 - 112
27 -68
21-89
15
EMC_NC_LD 15
I
liver segmentation. V-net and its trained model
on the same LiTS dataset are re-implemented
and based on the source code and introduction
3.2. Implementation
We implement the algorithms in Python 3
using Tensorflow 1.18 and CUDA 9.1. The
original source code for the FCN-CRF network,
and the trained model from [14] are reused and
modified to obtain a complete process of 3D
from
Chen’s
website
Tumor-Segmentation-Challenge. The DRIU
network model is fine-tuned using the
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
31
pre-trained model from Bellver et al [18]. The
parameter settings are the same as suggested in
the original work, including the batch size of 1;
15000 to 50000 iterations for a single channel;
the initial learning rate of 10 ; and SGD
optimizer with momentum.
The LCC method is implemented in Python 3,
using SITK library for connected components
extraction. For further studies, the source code for
the LCC method is publicly available at
The study utilizes a Linux PC, Ubuntu 16.04,
with Intel Core i9 9900K CPU, 8 cores, 3.6-5
GHz; NVIDIA Titan V GPU (11 GB RAM
version), 64 GB DDR4, 2133 MHz Bus.
4. Evaluation and result
4.1. Evaluation metrics
In this study, we assess the performance of
the combination of the CNNs with connected
components using several criteria introduced in
the MICCAI grand challenge. The algorithms
yield binary liver segmentations, which are
compared to the ground truth using Dice Score
(DSC), Mean Surface Distance (MSD),
Hausdoff Distance (HD), and False Positive
Rate (FPR). We also evaluate the processing
time of the methods. The evaluation metrics are
described in more detail below.
-8
Figure 3. Scores of the three CNNs with and without LCC on the three datasets.
The brief notations are described in the text.
4.1.1. Dice score (DSC)
(4)
( )
‖
‖
푑(푦, 푺 푿 ) = 푚푖푛푥∈푆(푋) 푦 − 푥 ,
Dice score is the overlap of the liver
segmentation and the ground truth. Given a liver
segmentation X and the ground truth Y, DSC can
be computed as:
‖ ‖
where . denotes the Euclidean distance.
MSD is then computed by:
1
(
)
∑
(
푥∈푆 푋
( )
푑푴푺푫 푿, 풀 =
( ) 푑(푥, 푺 풀 ) +
|
|
2 푿∩풀
| ( )| | ( )|
푆 푋 + 푆 푌
푫푺푪 =
,
(3)
|
|
푿∪풀
∑
( )
( ) 푑(푦, 푺 푿 ))
(5)
푦∈푆 푌
The maximum value of DSC is 1 when the
segmentation X is perfectly matched the ground
truth Y. The DSC is 0 when X and Y do not have
any voxel in common.
4.1.2. Mean Surface Distance (MSD)
Let S(X) denotes the set of surface voxels of
the segmentation X. The shortest distance of a
voxel y to S(X) is defined as:
4.1.3. Hausdorff Distance (HD)
Let S(X) and S(Y) be two boundaries of liver
segmentation and ground truth, respectively. The
Hausdorff distance dHD(S(X),S(Y)) is the
maximum distance between S(X) and S(Y), and
is computed as follows:
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
32
(
)
scores higher than 80% for every dataset. For the
EMC_NC_LD dataset, each of the CNNs fails to
d푯푫 푺(푿), 푺(풀) =
(
)
max{supx∈S(X) infy∈S(Y) d x, y , supy∈S(Y)infx∈S(X) d(x, y)},
segment one of the images, achieving Dice
(6)
where sup represents the supremum and inf
denotes the infimum.
scores less than 50%. We use 50% to decide the
threshold for failed cases. Based on Table 3,
we can conclude that V-net + LCC perform the
best with the medians of the Dice scores larger
than 90%. Note that 90% Dice score is also
the threshold for success used in other
applications [22].
The minimum and maximum processing
times, corresponding to the image size, are also
reported in the last column of Table 3. Based on
the statistics, we can conclude that the
DRIU+LCC runs faster than V-net + LCC.
Furthermore, the LCC takes less than a second
for refining segmentations by the three CNNs on
average. The maximum total processing time
suggests the largest adding time that radiology
technicians may have to take into account when
they combine the methods to other processes.
Note that the CT images are cropped to reduce
the redundancy in a data preparation step (See
section 3.1 Clincal Data).
4.1.4. False Positive Rate (FPR)
FPR is used to quantify the false positive
segmentation i.e. the segmentation outside the
ground truth. Given the segmentation X and the
ground truth Y, FPR of the segmentation can be
computed as the following:
|
|
푿\풀
푭푷푹(푿, 풀) =
,
(7)
| |
풀
where |X\Y| denotes number of voxels in X
which do not overlap with Y.
4.2. Quantitative results
The median values of the evaluation scores
of the liver segmentation predicted by using the
three CNNs architecture combined with the LCC
algorithm are summarized in the Table 3. All
three of the CNNs successfully segment the liver
in the Mayo and the EMC_LD dataset with Dice
Table 3. Median values of evaluation scores of LCC combined with the three CNN architectures. The numbers in
brackets are quality of improvement compared to without using LCC. The last column are the minimum and
maximum processing times. The bold number that they are the best scores
Dataset
Mayo
Methods
DSC (%)
92.3 (2.1)
HD (mm)
MSD (mm) FPR (%)
Processing
time (s)
FCN+CRF+LCC
DRIU+LCC
Vnet+LCC
FCN+CRF+LCC
DRIU+LCC
Vnet+LCC
63.4 (172)
34.6 (21)
25.3 (91)
35.1 (114)
42.0 (106)
38.2 (105)
51.5 (62)
66.1 (66)
51.7 (60)
4.4 (0.6)
2.2 (2.3)
1.6 (1.2)
2.5 (15.7)
2.4 (12.2)
2.0 (8.2)
3.6 (12.1)
4.9 (4.9)
2.2 (1.9)
3.1 (0.8)
7-8.2
92.6 (2.4)
93.8 (3.4)
86.0 (8.3)
84.7 (3.2)
90.4 (1.9)
81.9 (2.4)
87.2 (1.6)
90.3 (4.1)
8.1 (0.1)
6.7 (3.9)
13.5 (12)
14.9 (4.7)
14.2 (3.2)
23.3 (3.5)
8.8 (2.4)
7.8 (6.6)
5.6 – 6.1
6.6 - 9.8
3.1 – 6.4
2.6 – 5.3
4.2 - 8.6
3.6 – 7.7
2.6 – 6.8
2.9 - 8.4
EMC_LD
FCN+CRF+LCC
EMC_NC_LD DRIU+LCC
Vnet+LCC
i
Figure 3 is a box plot of the segmentation
on Mayo dataset), VM_LC (Vnet with LCC on
Mayo dataset), FEL (FCN+CRF on EMC
Lowdose dataset), FEL_LC (FCN+CRF with
LCC on EMC Lowdose dataset), DEL ( DRIU
on EMC Lowdose dataset), DEL_LC (DRIU
with LCC on EMC Lowdose dataset), VEL
(Vnet on EMC Lowdose dataset), VEL_LC
(Vnet with LCC on EMC Lowdose dataset),
Dice scores of all of three CNNs on the three
datasets with and without applying the LCC
algorithm. The brief notations are descried as the
following: FM (FCN+CRF on Mayo dataset),
FM_LC (FCN+CRF with LCC on Mayo
dataset), DM (DRIU on Mayo dataset), DM_LC
(DRIU with LCC on Mayo dataset), VM (Vnet
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
33
FEN (FCN+CRF on EMC Lowdose Non-
contrast enhanced dataset), FEN_LC
of the CNNs with and without the connected
components method. The p-values of the t-tests
for the evaluations scores of the pairs
FM/FM_LC, DM/DM_LC, VM/VM_LV,
FEL/FEL_LC, DEL/DEL_LC, VEL/VEL_LC,
(FCN+CRF with LCC on EMC Lowdose Non-
contrast enhanced dataset ), DEN (DRIU on
EMC Lowdose Non-contrast enhanced dataset),
DEN_LC (DRIU with LCC on EMC Lowdose
Non-contrast enhanced dataset), VEN (Vnet on
EMC Lowdose Non-contrast enhanced dataset),
VEN_LC (Vnet with LCC on EMC Lowdose
Non-contrast enhanced dataset). We also
perform paired T-tests to assess the statistical
significance of the difference between the results
PEN/PEN_LC,
DEN/DEN_LC
and
VEN/VEN_LC are summarized in Table 4.
From Table 4, we can conclude that the LCC
algorithm statistically significantly improves the
segmentation results of all three CNNs
in general.
Table 4. P-values of the T-tests for the proposed method with the corresponding original CNNs:
The numbers are smaller than 0.05 indicating that the improvements are statistically significance.
Dataset
Methods
DSC
HD
MSD
FPR
FM/FM_LC
0.021
0.019
0.002
0.001
-3
-3
< 10
< 10
< 10-3
0.019
Mayo
DM/DM_LC
VM/VM_LC
0.002
0.040
0.001
0.014
-3
-3
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10
< 10-3
0.118
< 10-3
< 10-3
< 10-3
< 10-3
FEL/FEL_LC
DEL/DEL_LC
VEL/VEL_LC
FEN/FEN_LC
DEN/DEN_LC
VEN/VEN_LC
0.010
0.016
0.027
0.034
0.055
0.019
-3
-3
-3
-3
-3
-3
-3
-3
-3
-3
EMC_LD
EMC_NC_LD
p
The Figure 4 is an example of 3D liver
approaches on CT images when they are
combined with connected component
segmentations on a low-dose contrast enhanced
CT image. In the second column, the liver
segmentations by three CNNs include some false
positive segmentations (in blue), which are
eliminated by the LCC algorithm. Obviously, the
difference in segmentation from three networks
is not visible in the 2D view (right column). The
3D view in the first column visualizes the
difference between the liver segmentations and
the ground truth.
a
algorithm and the largest component in a post-
processing step. We either re-implement or reuse
the CNNs model trained with the LiTS dataset,
testing them with other three datasets from two
different medical centers with both standard and
low dose protocols with and without contrast
enhancement. Next, we apply the LCC algorithm
on the liver segmentations by the CNNs
approaches and quantitatively evaluate the
results using well-known criteria for liver
segmentation. Combination of the CNN
approaches with the LCC algorithm statistically
significantly improves the liver segmentation.
The 3D visualization in the Figure 4 shows the
5. Discussion
In this study, we investigate the
improvement in liver segmentation using CNNs
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
34
improvements in a segmentation example. We
also conclude that the FCN combined with
conditional random forest method does not fully
liver segmentation, and thus cannot get rid of false
positive segmentations connected with the main
part, or fill in missing parts. More advanced
segmentation methods, such as level set and
graph-cuts, may further improve the smoothing
on the surface of the liver, since they can embed
and model liver shape and curvature
information. Thus, the precise liver surface
segmentation needs to be further investigated.
Perhaps, subsequent studies may use data
sharing to utilize more data in the training stage.
While data sharing is currently challenging due
to administrative procedures and privacy
concerns, data-augmentation research directions
could help enrich the training data pools.
There are some limitations in our study.
First, we only use 10 contrast enhanced CT, 15
low-dose contrast enhanced CT, and 15 low-
dose non-contrast enhanced CT from two
medical centers for evaluating the methods.
Nevertheless, we assume that the images from
other medical centers will yield similar results as
those in this study. Second, the training dataset
for the CNNs does not include low-dose CT
images, resulting in poor performance with the
EMC dataset. However, while investigating to
improve the CNNs with more dataset in the
training stage is not the main purpose of our
research, we believe that adding low-dose CT
images may improve the segmentation results.
The improvement may be limited due to effects
of the low-dose noise on the image quality. A
noise removal CNN network combined with the
current CNNs may be a more effective approach
to improve the liver segmentation. Third, there
have been several other variants of CNNs for
liver segmentation that have achieved adequate
results [17,23-27]. However, as pixel
classification based methods, these CNNs may
contain mis-classification parts and may likely
benefit as well from post-processing methods
such as the LCC.
eliminate
the
isolated
false
positive
segmentation. This can be explained by the fact
that the CRF only examines inter-slice
correlation of the segmentations, while the liver
segmentation should be connected in 3D as one
organ. From Figure 3, we can also conclude that
the CNNs work better with the regular dose
contrast enhanced CT images while most
improvements by the LCC occur with the low-
dose CT image. This may improve when more
low dose images are included in the training
stage. We refrained from adding more data in the
training stage. In our opinion, while retraining
CNNs network is a very “expensive” way of
research, reusing the shared works and
improving the result using “inexpensive”
techniques is a reasonable approach to promote
research results to practical application.
We also can see from Table 3 and Figure 3 that
V-net combined with the LCC generally perform
better than other methods. This confirms findings
from Milletari et al. (2016) [13], which show that
3D segmentation approaches use inter-slice
information and thus may improve segmentation
accuracy. However, Table 3 also demonstrates that
the 3D nature of the V-net leads to more
computation time and requires more memory.
These factors may limit its potential to be used in
clinical practices that require very fast processing
such as intra operation of liver RFA. Note that in
our experiment, we already manually cropped the
liver volume to avoid the redundancy while current
CT scans in clinical practice may have hundreds of
slices. A fast, automatic liver detection method
may be beneficial for those cases to extract the
region of interest while reducing the processing
time. Although the LCC shows to be effective for
liver segmentation, it still presents challenges. The
LCC can only remove false positive
segmentations, which are isolated from the main
F
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
35
Figure 4. Example of 3D liver segmentations by the three CNNs on a low-dose contrast enhanced CT image.
The first raw is segmentations by FCN, the second one is by DRIU and the last one is by V-net. The first column
contains the liver segmentations using with LCC (green) and the ground truth (red), the second column illustrates
the raw liver segmentation from the CNNs (blue) overlapped by the segmentation after post processing, and the
last column is the final 2D liver segmentations on 2D CT slice of the liver.
contrast enhancement of low-dose image. The
quantitative evaluation results show that LCC
statistically significantly improves the liver
segmentation accuracy of the CNNs, while
maintaining the processing time of less than 10
seconds in total for all of the networks, including
the LCC processing time of less than a second.
In our study, we find that V-net combined with
the LCC achieves a Dice score of approximately
94%, which is comparable to other state of the
6. Conclusion
In this paper, we present our work on
improving liver segmentation for CNN based
approaches using LCC algorithm. Experiments
are performed with three well-known CNN
architectures and with retrained or reused trained
models. We evaluate three datasets from two
different medical centers with regular contrast
enhanced CT image and both contrast and non-
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
36
hepatocellular
Australian study, Scandinavian journal of
gastroenterology 50(5) (2015) 567-576.
carcinoma:
A
multicenter
art methods. We believe that with the current
development of CNN-based approach research, the
liver segmentation usingCNNs has a high potential
to be applied in the clinical practice soon.
[8] H.M. Luu, C. Klink, W. Niessen, A. Moelker, T.
Van Walsum, Non-rigid registration of liver CT
images for CT-guided ablation of liver tumors.
PloS one, 11(9) 92016) e0161600.
Acknowledgments
[9] G. Gunay, M.H. Luu, A. Moelker, T. Van Walsum,
S. Klein, Semiautomated registration of pre‐and
intraoperative CT for image‐guided percutaneous
liver tumor ablation interventions, Medical physics
44(7) (2017) 3718-3725.
[10] A. Gotra, L. Sivakumaran, G. Chartrand, N. Vu, F.
Vandenbroucke-Menu, C. Kauffmann, A. Tang,
Liver segmentation: indications, techniques and
future directions, Insights into imaging 8(4) (2017)
This work has been supported by VNU
University of Engineering and Technology
under project number CN 18.03. We would like
to thank Mayo Clinical for supporting us their
data. We also would like to thank NVIDIA for
their aid of a graphics hardware unit.
377-392.
0558-1.
[11] T. Heimann, B. Van Ginneken, M.A. Styner, Y.
Arzhaeva, V. Aurich, C. Bauer, F. Bello,
Comparison and evaluation of methods for liver
segmentation from CT datasets, IEEE transactions
on medical imaging 28(8) (2009) 1251-1265.
Reference
[1] K.A. McGlynn, J.L. Petrick, W.T. London, Global
epidemiology of hepatocellular carcinoma: An
emphasis on demographic and regional variability.
Clinics
223-238.
in
liver disease 19(2) (2015)
[12] O. Ronneberger, P. Fischer, T. Brox, U-net:
Convolutional networks for biomedical image
segmentation, In International Conference on
Medical image computing and computer-assisted
intervention, Springer, Cham, 2015, pp. 234-241.
[13] F. Milletari, N. Navab, S.A. Ahmadi, October,
V-net: Fully convolutional neural networks for
volumetric medical image segmentation, In 2016
Fourth International Conference on 3D Vision
(3DV) IEEE, 2016, pp. 565-571.
[2] M.
Mohammadian, N. Mahdavifar, A.
Mohammadian-Hafshejani, H. Salehiniya, Liver
cancer in the world: epidemiology, incidence,
mortality and risk factors, World Cancer Res J.
5(2) (2018) e1082.
[3] T.T. Hong, N. Phuong Hoa, S.M. Walker, P.S. Hill,
C. Rao, Completeness and reliability of mortality
data in Viet Nam: Implications for the national
routine health management information system,
[14] P.F. Christ, F. Ettlinger, F. Grün, M.A. Elshaera, J.
Lipkova, S. Schlecht, M. Rempfler, Automatic
liver and tumor segmentation of CT and MRI
volumes using cascaded fully convolutional neural
networks, arXiv preprint arXiv:1702.05970, 2017.
[15] P.F. Christ, M.E.A. Elshaer, F. Ettlinger, S.
Tatavarty, M. Bickel, P. Bilic, H. Sommer,
Automatic liver and lesion segmentation in CT
using cascaded fully convolutional neural
networks and 3D conditional random fields. In
International Conference on Medical Image
Computing and Computer-Assisted Intervention,
Springer, Cham, 2016, pp. 415-423.
PloS
one
13(1)
2018)
e0190755.
[4] T. Pham, L. Bui, G. Kim, D. Hoang, T. Tran,
M. Hoang, Cancers in Vietnam-Burden and
Control Efforts: A Narrative Scoping Review.
Cancer Control 26(1) (2019) 1073274819863802.
[5] M. Borner, M. Castiglione, J. Triller, H.U. Baer, M.
Soucek, L. Blumgart, K. Brunner, Arena:
Considerable side effects of chemoembolization
for colorectal carcinoma metastatic to the liver,
Annals of oncology 3(2) (1992) 113-115.
[6] K. Memon, R.J. Lewandowski, L. Kulik, A. Riaz,
M.F. Mulcahy, R. Salem, Radioembolization for
primary and metastatic liver cancer, In Seminars in
radiation oncology, WB Saunders. 21(4) (2011)
294-302.
[7] I. Gory, M. Fink, S. Bell, P. Gow, A. Nicoll, V.
Knight, W. Kemp, Radiofrequency ablation versus
resection for the treatment of early stage
[16] H. Meine, G. Chlebus, M. Ghafoorian, I. Endo, A.
Schenk,
Comparison
of
U-net-based
Convolutional Neural Networks for Liver
Segmentation in CT. arXiv preprint arXiv, 2018,
pp. 1810.04017.
[17] X. Li, H. Chen, X. Qi, Q. Dou, C.W. Fu, P.A.
Heng, H-DenseUNet: hybrid densely connected
H.H. Son et al. / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 36, No. 1 (2020) 25-37
37
UNet for liver and tumor segmentation from CT
volumes, IEEE transactions on medical imaging,
37(12) (2018) 2663-2674.
deformation
in
radiofrequency
ablation
interventions using image registration, Physics in
Medicine & Biology 63(17) (2018) 175005.
[18] M. Bellver, K.K. Maninis, J. Pont-Tuset, X. Giró-
i-Nieto, J. Torres, L. Van Gool, Detection-aided
liver lesion segmentation using deep learning,
ArXiv preprint arXiv:1711.11069, 2017.
[23] A.A. Novikov, D. Major, M. Wimmer, D. Lenis, K.
Bühler, Deep Sequential Segmentation of Organs
in Volumetric Medical Scans, IEEE transactions on
medical imaging, 2018.
[19] H.S. Hoang, C.P. Pham, D. Franklin, T. Van
Walsum, M.H. Luu, An Evaluation of CNN-based
Liver Segmentation Methods using Multi-types of
CT Abdominal Images from Multiple Medical
Centers, In 2019 19th International Symposium on
Communications and Information Technologies
(ISCIT), IEEE, September, 2019, pp. 20-25.
[24] Y. Huo, J.G. Terry, J. Wang, S. Nair, A. Lasko, B.I.
Freedman, B.A. Landman, Fully Automatic Liver
Attenuation
Estimation
combing
CNN
Segmentation and Morphological Operations,
Medical physics, 2019.
[25] N. Gruber, S. Antholzer, W. Jaschke, C. Kremser,
M. Haltmeier, A Joint Deep Learning Approach for
Automated Liver and Tumor Segmentation, ArXiv
preprint arXiv, 2019, pp. 1902.07971.
[26] S. Chen, K. Ma, Y. Zheng, Med3D: Transfer
Learning for 3D Medical Image Analysis, ArXiv
preprint arXiv, 2019, pp. 1904.00625.
[20] H. Samet, M. Tamminen, Efficient component
labeling of images of arbitrary dimension
represented by linear bintrees, IEEE Transactions
on Pattern Analysis and Machine Intelligence,
10(4) (1988) 579-586.
[21] P. Bilic, P.F. Christ, E. Vorontsov, G. Chlebus, H.
Chen, Q. Dou, S. Kadoury, The liver tumor
segmentation benchmark (lits), ArXiv preprint
arXiv, 2019, 1901.04056.
[22] H.M. Luu, A. Moelker, S. Klein, W. Niessen, T.
Van Walsum, Quantification of nonrigid liver
[27] W. Tang, D. Zou, S. Yang, J. Shi, DSL: Automatic
Liver
Segmentation
with
Faster
R-CNN and DeepLab, In International Conference
on Artificial Neural Networks, Springer, Cham,
2018, pp. 137-147.
O
-
13 trang
12600
Free
Bạn đang xem tài liệu "Liver segmentation on a variety of computed tomography (CT) images based on convolutional neural networks combined with connected components", để tải tài liệu gốc về máy hãy click vào nút Download ở trên
File đính kèm:
- liver_segmentation_on_a_variety_of_computed_tomography_ct_im.pdf
