Review on the Methods of Automatic Liver Segmentation from Abdominal Images

Automatic liver segmentation from abdominal images is challenging on the aspects of segmentation accuracy, automation and robustness. There exist many methods of liver segmentation and ways of categorisingthem. In this paper, we present a new way of summarizing the latest achievements in automatic liver segmentation.We categorise a segmentation method according to the image feature it works on, therefore better summarising the performance of each category and leading to finding an optimal solution for a particular segmentation task. All the methods of liver segmentation are categorized into three main classes including gray level based method, structure based method and texture based method. In each class, the latest advance is reviewed with summary comments on the advantages and drawbacks of each discussed approach. Performance comparisons among the classes are given along with the remarks on the problems existed and possible solutions. In conclusion, we point out that liver segmentation is still an open issue and the tendency is that multiple methods will be employed together to achieve better segmentation performance.


Introduction
The liver is one of the most important organs of human body.Liver diseases result heavy death worldwide.In helping doctors and surgeons treating liver diseases, computer aided liver disease diagnosis and liver surgical planning systems are playing important roles nowadays.An accurate and automatic segmentation approach of liver parenchyma, vessel and tumour is crucial to a computer-aided liver disease diagnosis and liver surgical planning system such as a system for liver transplantation.However, due to the highly varying shape of liver, low contrast and intensity in homogeneity inside liver, weak boundaries to its adjacent organs such as heart and stomach, and intensity homogeneity to adjacent organs, liver segmentation becomes a challenging task that has attracted much research attention recently.
Liver segmentation from abdominal images is a process of subdividing a medical image such as computed tomography (CT) and magnetic resonance imaging (MRI) into liver parenchyma area and non-parenchyma areas.There exist many methods of liver segmentation including region growing, active contour, level set, graph cuts, clustering and threshold based methods, deformable model, statistic shape model; support vector machine (SVM) based, neural network (NN) based, etc.Some comprehensive reviews have been done on liver segmentation [1][2][3][4].It is noticed that a particular categorising methodology is adopted to well address an emphasis in these reviews.For example, [5] reviews the machine learning techniques for automatic segmentation of liver images, where the techniques are classified as NN based, support vector machine-based, clustering based, and hybrid technique; [1] surveys segmentation of liver from CT images, where the techniques are classified as region based, threshold based, level set, model based, active contour based, histogram based, gray level based, and clustering based.
Based on thorough study of various liver segmentation methods and systematic summary of the methods, we categorise a segmentation method according to the image features it works on, therefore better summarising the performance of each category and leading to find an optimal solution for a particular segmentation task.All the methods are categorized into three main classes including gray level based method, structure based method and texture based method.This paper is organized as below.
Section 2 presents the literature review of liver segmentation.It is structured in three categories including gray level based method, structure based method and texture based method.Section 3 discusses performance comparisons among the classes along with the remarks on the problems existed and possible solutions.Finally, conclusions are made in Section 4.

Review
The latest achievements in automatic liver segmentation are reviewed in this section.All the methods are discussed in one of the three categories including gray level based, structure based and texture based.

Gray Level Based Methods
Gray level is the most obvious feature of image.When extracting objects from image, the most natural way is to use the gray level to tell boundaries.The benefits of gray level based methods are: the feature is easy to extract without using special algorithm; they are stable and robust, can easily be used into similar cases; they often achieve high accuracy result.Their drawbacks are: most of them are semi-automatic methods and need user's operation; when the difference of gray level intensity between target and background is small, the methods will lose their effectiveness.This section reviews various gray level based liver segmentation methods.

Region Growing
The seeded region growing algorithm was first proposed by [6].It puts the pixels with the similar gray value together as a same region.Comparing with the histogrambased region selection, the region growing can use more features rather than just gray level intensity.The average intensity, variance, color or texture etc. all can be used to measure the similarity of pixels or small regions.The drawbacks are: the seed point should be chosen by user; it will lose the effectiveness when the region is inhomogeneous.
The efficiency of these methods depends on the choice of the seed points strongly.The seed points are usually selected by users to improve the quality of segmentation.[7] presented a performance benchmarking study on liver tumour segmentation for three semi-automatic algorithms: 2D region growing with knowledge-based constraints (A1), 2D voxel classification with propagational learning (A2) and Bayesian rule-based 3D region growing (A3).Its conclusion is A1 and A2 is more effective than A3.Some other methods are fully automatic.[8] presented an adaptive region growing method that learns its homogeneity criterion from characteristics of the region to be segmented automatically.However, the efficiency of this method depends on the homogeneity of the tissue and cannot avoid the under-segmentation when the target is inhomogeneous.[9,10] based on [8]'s work and made some improvement to deal with the inhomogeneity.They determined the seed region based on the intensity of gray level; and then separated the liver and heart based on the anatomical feature; next, used an improved region growing to segment the image; finally, employed a postprocessing to deal with the under-segmentation near the right lung lobe.Their method can deal with most cases well and it is quick; but in some difficult cases (e.g. the gray level intensity of live is inhomogeneous because of large lesion), it will cause under-segmentation.In the work of [11], the centroid of the largest connected region of the eroded image is determined and the coordinates of centroid point acts as the initial seed point for region growing; then a gaussian model was used to determine the threshold range of region growing; finally, a postprocessing was employed to fix the hole and the connection with other neighbourhood tissues.It can achieve good results and less time-consuming.[12] employed ray casting algorithm as a pre-treatment to transform the initial image into a projection plane, and then used region growing to segment liver from CT scans.

Active Contour
Active contours are curves that deform within digital images to recover object shapes.They are classified as either parametric active contours or geometric active contours according to their representation and implementation.In particular, parametric active contours are represented explicitly as parameterized curves in Lagrangian formulation.Geometric active contours are represented implicitly as level sets of two dimensional distance function which evolve according to an Eulerian formulation [13].
Parametric active contours (active contours, or snakes) are curves defined within an image domain that can move under the influence of internal forces coming from within the curve itself and external forces which are defined so that the snake will conform to an object boundary or other desired features within an image.One successful implementation is gradient vector flow (GVF) [14].The main advantage is that it can capture the targeted object without a great definition of the initial boundary, which can be iteratively calculated in-and outwards.It can find simultaneously both concave and convex features.[15] presented a new model (vascular active contour model) to deal with vessel segmentation.It can drive the active contour into the thinner and weaker vessel regions.And a dual curvature strategy is added to smooth the surface of the vessel without changing its shape.Active contour are also used to refine the segmentation results and improve the accuracy [16,17].
Geometric active contour are based on the theory of curve evolution and the level set method.Level set methods focus on the gradient of gray level, leads the boundary to approach the position where gradient is maximum.The basic idea of the level set method is to represent a contour as the zero level set of a higher dimensional function, called level set function, and formulate the motion of the contour as the evolution of the level set function.It has been successfully used in the segmentation of medical images.The main advantage of level set is that it allows changes of surface topology implicitly; however, it is time-consuming, and often semi-automatic, and could lead to over-segmentation.To increase the computation speed and make the method fully automatic, [18] splatted the method into two steps: first is a rough segmentation by employing fast-marching level set and then a geodesic active contour level set algorithm was used to refine the initial approximation.
There are also other ways to achieve the rough segmentation.[19] presented a new fuzzy level set algorithm.It begins with spatial fuzzy clustering.The result was utilized to initialize level set segmentation and estimated the parameters of level set evolution.Moreover, the fuzzy level set algorithm was enhanced with locally regularized evolution which can facilitate level set manipulation and lead to more robust segmentation.It is efficient when the background is simple and the boundary between background and object is clear.In many cases, the segmentation result of one slice could be used as an initial segmentation of the adjacent slice [20,21].To achieve better segmentation results, the original level set algorithm could be optimized.[22] proposed a level set based variational approach that incorporates shape priors into edge-based and region-based models.The energy function took smoothness of the region and the length of boundary into account, made the description of the boundary shape more specifically.[23] presented an optimized level set algorithm.Instead of using the same parameter values in all stages of the algorithm, they proposed to change them depending on the resolution step by means of a multi-curvature, multi-growth strategy and a fine detail correction at the last multi-resolution level.

Graph Cuts
The main idea of graph cuts is to represent the image to an undirected weighted graph.Every node represents each pixel of image.Every edge connected a pair of adjacent pixels.The weight of edge indicates the similarly of gray level, colour or texture between each pair.The segmentation is a cut of the graph.Each region represents a subgraph.The best cut is to make the similarity in a subgraph maximum and the similarity between subgraphs minimum.It has become increasingly popular in medical image segmentation within recent years because it is not iterative, and achieves global minimization for certain classes of energy functions.However, it is not a fully automatic method because it needs users to select seed points which label the "object" and "background".The applications on liver segmentation often focus on the vessel and tumour.The reason is that when segmenting the vessel or tumour, the "background", liver, is homogeneous.It is easy to extract the object from a simple and homogenous background by using graph cuts method.When using it to the segmentation of liver parenchyma, the seeds of background should include every other region, which is difficult.
In liver parenchyma segmentation, the seed points should be selected carefully.To make the method fully automatic, many algorithms were employed to select seed points.Fast marching and mathematical morphology were utilized in [24] to achieve a rough segmentation of liver and background of CT image.Statistic adaptive threshold initialization [25] and k-means clustering [26] can also achieve the same purpose.However, compared with other gray level based methods, graph cuts used in liver parenchyma segmentation show more difficulties.

Threshold Based
The segmentation methods which base on threshold are often used to make a rough segmentation to determine ROI or seed points as pre-processing [10][11][12].Firstly, the intensity of initial image is enhanced; then the range of liver or tumour is determined based on histogram analyse and prior knowledge; finally, the rough segmentation is completed based on the determined threshold.In liver parenchyma segmentation, the threshold based methods are too rough to achieve good result.However, it could be employed to segment tumours from liver since the contrast between liver and tumour is more significant.[27,28] used threshold based method as the main segmentation method of tumour.First, the contrast of gray level intensity was enhanced; second, the slice was added to itself, after that, the contrast between liver and tumour is large enough to use the threshold to isolate tumour directly; thirdly, the threshold based method was used to segment the image; finally, the morphological filter was employed as the post-processing.Since it is sensitive to noise, [27] used roundness and information of neighbouring slice to reduce false detection.

Clustering Based
The main idea of clustering based method is that in n-dimensional feature space, the distance between samples is shorter if they belong to the same class and the similarity of samples from same class is higher.There are two main issues in clustering based methods: one is how to estimate the similarity of samples, the other is how to determine the threshold of similarity.The advantages of clustering based methods are: they are fully automatic and they can handle multiple tasks of segmentation.However, its results may contain many false positive region, needing post-processing.
The clustering based methods used on liver segmentation generally include two classes: Fuzzy c-means (FCM) clustering and k-means clustering.In FCM, each point has a degree of membership to classes rather than belongs to just one class completely.Points on the edge of a class may have less membership degree than points in the centre of class.When the number of classes is given as n, all points will be classified into n classes based on the membership degree and Euclidean distance between each point and class centre.In [29][30][31] the initial image was segmented by fuzzy c-means clustering and then smoothed by morphological processing.Then the candidate regions were analysed based on computing properties [29] or classified by neural network [30,31].Finally, the regions which belong to liver or node were extracted.Compared with k-means clustering, FCM can successfully segment the liver from CT images, despite the similar gray level of adjacent organs and different gray level of tumours in the liver.Also, the FCM can be used to refine the rough segmentation [32].K-means clustering does not often act as main segmentation method since it is too rough.[26,33] employed k-means clustering to select the object and background seeds for graph cuts.[34] used the k-means clustering result as the initial boundary of active contour.
There are also other clustering based methods.Hierarchical agglomerative clustering combine with selforganized map was utilized to achieve liver segmentation of MR image [35].The EM/MPM clustering method which exploits both intensity of voxels and labels of the neighbouring voxels was employed to cluster liver tissue [36].

Structure Based Methods
Structure based methods have proven to be effective and powerful in many medical applications.The central hypothesis of it is that structures of interested objects have a repetitive form of geometry.In the approach, a probabilistic model is created to represent the variation of the shapes of organs, and use this model as prior knowledge to impose constraints in an image for segmentation.
[37] presented a semi-automatic segmentation method by using deformable model.Statistical shape model (SSM) is used for liver segmentation frequently because of its ability to constrain the segmentation to approximately match previously seen shapes of a training data base.[38] proposed a hybrid liver segmentation method using statistical pose model (SPM) and probabilistic atlas.It achieved an average segmentation score of 72.4.[39] presented an approach for automatic liver segmentation based on SSM integrated with an optimal-surface-detec-tion strategy.The method achieved higher accuracy comparing to previous model based methods.[40] presented a novel statistical shape model approach for fully automatic CT liver segmentation.The method combines learned local shape priors with constraints that are directly derived from the current curvature of the model in order to restrict adaptation to regions where large deformations were expected and observed.That makes the method more robust and reduces the leakage to other organs.[41] used the model information to choose a segment with the highest fidelity to the organ instead of using the model information to direct the segmentation.

Texture Based Methods
Texture based methods are different from other segmentation methods.They do not focus on the boundary of object.Instead, they are interested in the texture features.The main procedure of texture based methods is: firstly, the texture features of target are extracted; then a classifier is employed to classify the features; finally, the target region is refined and smoothed by post-processing.
Many different types of texture features are used in liver segmentation.[42] extracted 4 features from liver CT image including neighbourhood mean, neighbourhood variance, Law's texture, and Unser's sum-and-difference histograms.[43] used the appearance feature and context feature to describe liver in 3D CT images.[44] listed many tumour features such as tumour volume, tumour diameter, tumour sizes region ratio, etc.It employed the minimum redundancy and maximum relevance as feature selection method which selects useful features according to the maximal statistical dependency criterion based on mutual information and minimizes the redundancy among features simultaneously.Wavelet coefficients were used to extract texture characteristics of liver and its surrounding tissues [45].[46] presented an accurate liver segmentation algorithm which used texture features including high order statistical texture features and anatomical structural features.A feature extraction method of liver tumour based on watershed was employed [47].
Many classifiers are employed to determine liver target.As a classifier, support vector machine (SVM) does not need much training data to achieve a good result.It is appropriate for the liver segmentation.[44] presented a SVM based liver segmentation method using wavelet features and 3D median filtering as post-processing.[45] used SVMto classify the data into pixel-wised liver area or non-liver area, then integrated morphological operations were used to remove noise and finally delineated the liver.Watershed transform was used in [47] to extract features of liver tumour and SVM was used to make the classification.The connected region detection and morphological operations were employed as post-process-ing.
Neural networks (NNs) are non-linear statistical data modelling or decision making tools.They can be used to model complex relationships between inputs and outputs or to find patterns in data, which makes it suitable for liver segmentation.[31] employed NN to classify the result of fuzzy c-mean clustering, identify which regions belong to liver tissue.The same method was also used [33] to identify the tumours from liver.Watershed algorithm was used in [48] to over-segment the MRI into many small regions, and NN was used to get the feature value of liver image.By comparing the feature value of each small region with the output of NN, the liver region was selected and the segmentation parameter was adjusted, repeated the segmentation until the difference between feature value of liver region and NN output did not decrease.[49] used the colour information of liver MR image in three colour spaces, then used Hopfield neural network (HNN) to segment the tissue of liver.
Genetic algorithm [20,21] and extreme learning machine [42] can also be used in liver image segmentation by combining with other methods.
With the development of machine learning technology, more texture based methods will be employed in the image segmentation area.

Discussion
All the three categories of liver segmentation methods have their own merits and drawbacks and may be effective for a particular case.In this section, their characteristics are summarized.
The gray level based methods directly utilize image's features.They are the main methods used in clinical practice, especially in tumour segmentation.However, these methods rely heavily on the evaluation of the gray level of targets.Some methods use prior knowledge to determine the gray level range of the liver.They are fast, but may lose their effectiveness when the gray level of the target changes.Some other methods utilize histograms to estimate the gray level of the liver.They avoid the problems of using prior knowledge, but may fail when the image represents a low percentage of the liver.There are also some methods that use manual work or automatic rough segmentation to select a small region of the liver and based on this region to compile information about the gray level.They are more reliable, but need manual work or more computation time.Some gray level based methods use gradient information.The advantage of these methods is that they are sensitive to boundaries, resulting exactly approaches to image boundaries.The disadvantage is that there are many boundaries in the image, only a few of which are the real boundaries of the liver.These methods will easily converge on the stronger fake boundaries, causing over or under segmentation.To refine the result, manual work or other methods are needed.
The structure based methods can deal with the unclear boundary of the liver by using prior knowledge, meaning that they can handle some problems which gray level based methods cannot handle.The difficulty of these methods is that they need a large amount of training data to cover all the conditions of the liver.It is even more difficult with liver's non-standard shape, which makes it hard to describe liver using a unified model.This is the main limitation for the approach in clinical practice.
Using texture based methods to segment is more like using human eyesight to do segmentation.Just as humans use texture information to determine boundaries rather than gray level or shape, texture based methods often rely on machine learning and pattern recognition.The advantages are that more features are considered together, and the result is closer to the results of manual segmentation.Similar to the structure based methods, texture based methods can also achieve better results when the boundaries are not clear.In addition to the need of training data, the description of texture feature is a challenge.Although there have been many descriptors, these descriptors are not like those described by human.In addition, there are so many kinds of descriptors that selection among them is a problem.Machine learning and pattern recognition are still developing technologies with much weaker information processing abilities than human brain.They cannot currently produce satisfactory segmentation results, making it necessary to find more refined methods.
In general, the gray level based methods are more highly developed.They are often used together to handle the problem of complex segmentation.In most cases, they can achieve better segmentation results.Structure based methods focus on the shape of the object, which makes them more robust.Texture based methods try to simulate the way our brains process information.

Conclusion
This paper reviews automatic liver segmentation from abdominal images.The review is structured in a new way of categorising a segmentation method according to the image feature it works on, therefore better summarising the performance of each category and leading to find an optimal solution for a particular segmentation task.All the methods of liver segmentation are categorized into three main classes including gray level based method, structure based method and texture based method.Performance comparisons among the classes are given along with the remarks on the problems existed and possible solutions.In conclusion, we point out that liver segmentation is still an open issue and the tendency is that multiple methods will be employed together to achieve better