3 D segmentation and visualization of lung and its structures using CT images of the thorax

Computing systems have been playing an important role in various medical fields, notably in image diagnosis. Studies in the field of Computational Vision aim at developing techniques and systems capable of detecting various illnesses automatically. What has been highlighted among the existing exams that allow diagnosis aid and the application of computing systems in parallel is Computed Tomography (CT). CT enables the visualization of internal organs, such as the lung and its structures. Computational Vision systems extract information from the CT images by segmenting the regions of interest, and then recognize and identify details in those images. This work focuses on the segmentation phase of CT lung images with singularity-based techniques. Among these methods are the region growing (RG) technique and its 3D RG variations and the thresholding technique with multi-thresholding. The 3D RG method is applied to lung segmentation and from the 3D RG segments of the lung hilum, the multi-thresholding can segment the blood vessels, lung emphysema and the bones. The results of lung segmentation in this work were evaluated by two pulmonologists. The results obtained showed that these methods can integrate aid systems for medical diagnosis in the pulmonology field.


INTRODUCTION
A large number of diseases that affect the world's population are lung-related.Therefore, research in the field of pulmonology has great importance in public health studies and focuses mainly on asthma, bronchiectasis and Chronic Obstructive Pulmonary Disease (COPD) [1,2].
The World Health Organization (WHO) estimates that there are 300 million people who suffer from asthma, and this disease causes around 250 thousand deaths per year worldwide [3].Also, WHO estimates that 210 million people have COPD.This disease caused the death of over 300 thousand people in 2005 [4].Recent studies reveal that COPD is present in the 20-to 45-yearold age bracket, although it is characterized as an over-50-year-old disease.Based on this, WHO estimates that the number of deaths due to COPD will increase 30% by 2015, and by 2030 COPD will be the third cause of mortalities worldwide [5].
During the period from 1992 to 2006, 15% of all hospitalizations financed by the Brazilian Federal Health System (SUS) were due to pulmonary diseases, of which asthma and COPD together summed up 562,016 hospitalizations [3].
Thus, it is of fundamental importance for the public health system to obtain an early and correct diagnosis of any pulmonary disease.Diagnosis aid is important from a clinical point of view as it increases the amount of information the specialist has concerning the patient's state of health.Therefore, certain illnesses can be detected precociously, even saving lives in some cases.Besides, some techniques allow the clinical image of the disease to be tracked appropriately [6,7].
The segmentation stage pulmonology systems are essential for the correct and accurate medical diagnosis aid, as this stage delimits the lung area in CT images of the thorax, which must be analyzed by the system or by a specialist.
The segmentation of objects and structures in medical images is a process that, in most cases, is more complex if compared to the segmentation of other image types, such as metallographic and synthetic aperture radar remote sensing.This occurs due to the variability of structures and/or internal organs, the visualization plane of these images and furthermore, the possibility of diseases affecting these organs.All of which contribute to increasing the difficulty to develop efficient techniques for medical image segmentation [8].
CT images of the lungs represent a slice of the ribcage, where a large number of structures are located, such as blood vessels, arteries, respiratory vessels, pulmonary pleura and parenchyma, each with its own specific information.Thus for analysis and for the pulmonary disease diagnosis aid, it is necessary to segment the lung structure.
Lung segmentation techniques have been developed seeking optimization at this stage.Sluimer, Prokop and Ginneken in [9], and Felix et al. in [6], achieved lung segmentation of CT images using the region growing method.Felix et al. in [10], Hu, Hoffman and Reinhart in [11], and Silva et al. in [12], put together mathematical morphology, region growing, thresholding and edge detection methods to obtain even more accurate results.The Pulmonary Emphysema Detection system (PEDS) performs automatic lung segmentation through region growing associated with binary morphological operations of dilation and erosion [8].
Another methodology for lung segmentation is the 3D region growing approach, and this is applied to segment the lung and its internal structures, such as the vessels and airways [13][14][15][16].This method works similarly to the 2D region growing technique that is initialized by a seed point and then expanded through its neighbors, obeying an aggregation rule.
In addition, other computational tools have also been used to evaluate lung structures, for example, Chen et al. in [17], compared the diagnostic performances of an artificial neural network (ANNs) and multivariable logistic regression analyses for differentiating between malignant and benign lung nodules on computed tomography scans.Er, Yumusak and Temurtas in [18], also used ANNs to analyze chest disease diagnoses by using multilayer, probabilistic, learning vector quantization, and generalized regression neural networks.These authors also used an artificial immune system classification [19], and obtained high accuracy in chest disease diagnoses.Fernández et al. in [20], evaluated serum biomarkers levels in lung cancer patients and non-cancer controls using principal component analysis and ANNs modeling.
Active contour methods have also been widely reported in the literature to segment lung structures from CT images.Annangi et al. in [21], proposed a regionbased active contour method for x-ray lung segmentation using prior shape and low level features.Wu, Wang and Jia in [22], presented a novel external force, called adap-tive diffusion flow (ADF), with adaptive diffusion strategies according to the characteristics of an image region in the parametric active contour model framework.Keshani et al. in [23], used a support vector machine combined with an active contour model to segment lung nodules.Tan, Schwartz and Zhao in [24], combined image processing techniques of marker-controlled watershed, geometric active contours as well as the Markov random field to evaluate lung lesions considering their size, density, and shape.Finally, Liu and Bovik in [25], proposed a novel external force for active contours, which we call neighborhood-extending and noise-smoothing gradient vector flow to extract the parenchyma and cancer regions.
This work contemplates two singularity methods, the 3D region growing and the multi-thresholding algorithms, for the segmentation of the lung and its internal structures.The methods aim to assist medical diagnosis aid in pulmonology diseases though 3D visualization and reconstruction from bi-dimensional CT images.After image segmentation, it is possible to visualize and quantify the lung structures, as well as its internal structures tridimensionally.This facilitates the diagnosis by the medical experts and reduces/eliminates the subjectivity of interpretation of the test.The results obtained from the proposed method that performs the bi-dimensional segmentation and volumetric quantifying of all lung structures were evaluated and validated by two medical specialties (pulmonologists).
One of the main contributions of this work is to use the proposed method on personal computers, maintaining, or in some cases, improving the performance.Also the use of a personal computer reduces the processing costs, the use of the CT scanner workstation and reduces the time to evaluate a test.

MATERIALS AND METHODS
This section describes the digital image acquisition through Computed Tomography (CT) and, afterwards, the lung segmentation, reconstruction and visualization method using the 3D region growing method.
Prior to the image acquisition, the tomograph is set to an air density of −1000 HU (Hounsfield Units).The caliation is carried out within three months of the exams, as specified by the manufacturer [26].Also before the acquisition, a tomographic cut with a water phantom with a known density is performed for analysis and parameter control of the system [2].The images are quantified in 16 bits and stored in a DICOM format (Digital Imaging and Communications in Medicine).
To read these image files, which are in the DICOM format, the free DICOM toolkit offered by OFFIS was used.This library is compatible with the C ++ programming language, as used in this work.

Figure 1(a)
shows an example of an image obtained using CT with a multi-detector.This image is based on the principle that every internal structure of the human body possesses a determined radiologic density value, expressed in Hounsfield Units.The pulmonary densities adopted in this work are: hyper aerated (−1000 to −950 HU), normally aerated (−950 to −50 HU), poorly aerated (−500 to −100 HU), not aerated (−100 to 100 HU) and bone region (600 to 2000 HU) [27,28].The distribution of these density bands is shown in Figure 1(b), in which the red color represents the hyper aerated region, the dark blue the normally aerated areas, the light blue represents the poorly aerated areas, and the black the non aerate areas, while the green represents the bone region and the white the non-classified area.

3D Pulmonary Visualization and Reconstruction
The graphic visualization system used in this work was the Open Graphics Library (OpenGL) Application Programming Interface (API).This API is an open and multiplatform specification of a graphical and modeling routine library, for the development of graphical computing applications, such as games and visualization systems.Two libraries that make up the OpenGL, GLU and GLUT, both open sources were used in this work.The OpenGL Utility Library contains various routines with low level OpenGL commands to execute tasks such as defining the matrix for the projection and orientation of the visualization, and render a surface.The OpenGL Utility Toolkit (GLUT), which is a non-platform dependent toolkit and includes some graphic interface elements was also used here.
Thus the routines for the 3D development system of this work were developed in C/C ++ language, and called for the OpenGL library routines.The GLU library is applied to render objects, and configure their shape and illumination.The GLUT library, was also used to create windows and receive the user commands.This system is capable of being visualized in various platforms.
In the case of specific CT images, the image spacing, or between planes, is one of the DICOM pattern properties, defined by the slice thickness property.The pixel size is application specific when these images are 3D processed and in the DICOM pattern, and this is determined by pixel spacing.Using these two parameters it is possible to reconstruct the CT on the same scale it was generated on.

Lung Segmentation Using 3D Region Growing
Region growing is an image segmentation technique used to unite the regions of interest.This technique clusters the subgroups or groups of pixels in a determined region.This is possible through the expansion of a given region initialized by a pixel, named seed.This expansion occurs by the aggregation of pixels to the region through successive analysis iterations of the neighborhood of a given region [29].
In this technique, for a pixel to be grouped to the region, it is necessary for it to be in the region's neighborhood and to obey pre-established criteria.This generally bases itself on predefined parameters such as intensity of gray tone, and gray tone mean in the region, among others.
An illustration of the technique's application is presented in Figure 2, in which the seed is shown in red, Figure 2(a), and the clustering ruling is that the gray tonality needs to be equal to the chosen seed's tonality, resulting in the shown segmentation in Figure 2(b).
The efficacy of this method depends directly on the choice of the seed and the grouping rule.If these choices are not adequate, the segmentation could present failures.The 3D expansion of this technique is possible assuming each element's neighborhood is analyzed considering nearby planes.This technique is common for objects and medical image segmentations, principally in CT images.A set of images from such exams follows a predetermined sequence, obeying a standard pattern [13,16].In these cases the smallest element is the voxel and the locale begins being analyzed in the "x", "y" and "z" axes.An example of this evolution is presented in Figure 3.The block diagram to execute the 3D region growing method is presented in Figure 4, and its initialization at the curve startup is responsible for the segmentation of the regions of interest.
The algorithm flowchart is presented in Figure 4; the first step of the method is the initialization with the seed.In this work the initialization through the INAUTO, in [8], method is employed, in which all images are analyzed and when two seeds are found in regions which contain lungs (−500 to −1000 HU), the position of the "z" axis of these targets is stored.The mean "z" coordinate among all previously stored targets is used as the initialization target and the points found through the method are the seeds of the 3D region growing method.Figure 5 shows the 3D RG Steps.
From the initialization the region grows by successive iterations, calculating the area before, analyzing and aggregating the predetermined neighboring region.At the end of every iteration, the area is recalculated.The chosen method of analysis and the clustering of neighboring regions utilize the pulmonary anatomy information, adding voxels that are in some intensity band within the lung, which are: normally, poorly or hyper aerated (−500 to −1000 HU).This adding is made by successive   iterations until stabilization when no voxel is aggregated to the region.

Thresholding and Multi-Thresholding
This segmentation technique is based on thresholding.Its basic principle is to determine a value as a threshold, generally in a gray tone that is within the range of tones used in the image.For example, in an image with an 8 bit resolution, the threshold may be between 0 and 255.
After establishing the threshold of all the regions in the image, it is possible to label every pixel, associating it to the value band established in each region.When there are just two regions for classification, one of these receives the label 0 and the other 1, and in this case the technique is called binarization.More than one threshold can be established in the same image; this technique is called multi-thresholding [30,31].This technique subdivides the image in more than two regions, establishing the lower and the higher limits of each region of interest.The multi-thresholding was used in this work with pulmonary anatomy data, in which every human tissue obeys a specific density bracket.The main density brackets being: 1) hyper aerated (−1000 to −950 HU), 2) normally aerated (−950 to −500 HU), 3) poorly aerated (−500 to −100 HU), 4) not aerated (−100 to 100 HU), and, finally, 5) bone (600 to 2000 HU).
Using these radiological density brackets, it is possible to detect some structures and diseases in the lungs such as emphysema, vessels, airways and bones.From these, bones can be detected and located without the lung segmentation, while the rest of the structures are detected and located after the lung segmentation.

Qualitative Evaluation Measurements
The evaluation measures adopted in this work are based on the qualitative approach of the segmentation as proposed by Gonzales and Woods in [29].Thus, there are 5 classification criteria considered in the evaluation criteria: optimal, acceptable, reasonable, bad and worst.This classification and its respective grades are described in Table 1.
This classification, presented in Table 1, was used by Rebousas Filho in [32], in the evaluation of lung segmentations of CT images.Also, Cavalcante in [33], evaluated the segmentation of airways by this qualitative evaluation method.

Inter and Intra Observer Evaluation Measurements
The index of inter and intra concordance constitutes a measure used to analyze the agreement between two observation intervals (intra observers) and between evaluators (inter observers) in the attribution of categories for a determined classification, as shown in Table 1.
The statistical concordance measurement used in this work is the Cohen's Kappa index.This index is an intra and inter observer concordance measurement and measures the degree of concordance between two observers and varies from 0 to 1, in what 1 means a greater concordance and 0 suggests the concordance is attributed randomly [34].The Kappa index is calculated from a confusion matrix, an example of this is presented in Table 2.
The Kappa analysis is a discrete multivariable used in a thematic precision evaluation and uses all the elements of the confusion matrix.The Kappa coefficient (K) is a measurement of the real concordance, indicated by the diagonal confusion matrix elements, minus the concordance by chance, indicated by the total line versus the column result, which does not include non-recognized inputs.This means it is a measure of how much the classification is in accordance with the reference data.Further details on the calculation of the kappa index can be seen in [35].
Although the Kappa coefficient is quite often used to evaluate the exactness of mapping, there are no minimal acceptable levels of this coefficient in a classification [35].However, Table 3 presents classification performance levels, normally accepted by the scientific community.
Nowadays, the Kappa index is largely used to calculate the concordance level between observers in various applications such as, classification and image analysis of Synthetic Aperture Radar [36], classification of subcellular molecules [37], classification of breast lesions [38] and the evaluation of multimedia quality [39].[35].
Line total n +i X +1 X +2 X +c n

RESULTS AND DISCUSSION
In this section, the method of medical image acquisition is presented first.Then, the segmentation using the 3D region growing is evaluated on CT lung images of the thorax of healthy patients and patients with COPD and fibrosis.Finally, some applications are presented to prove the robustness, effectiveness and efficacy of the proposed method.

Acquisition of the Medical Images
The images used to evaluate the algorithms were acquired from different CT scanners, some of the images were saved as samples of a complete CT exam, while other images were saved together with the complete exam.
The models used to acquire the complete exams were the Toshiba Aquilion (TA), the GE Medical system LightSpeed 16 (GEMSL) and the Phillips Brilliance 10 (PB).All images were of 512 × 512 resolution and 16 bits.Table 4 gives the characteristics of these complete  exams, conceded by a pulmonologist, with the patient's authorization.These images constitute an image cluster obtained in partnership with the Walter Cantídio Hospital of the Federal University of Ceará, submitted to an earlier study [1,2].This study was approved and evaluated by the UFC Research Ethics Committee-COMEPE (Protocol n˚ 35/06) and complied with the demands of Resolution n˚ 196/96 of the National Health Council, concerning research in human beings.

3D Region Growing
In the tests 11 complete thorax CT exams were used.The exams were taken of healthy patients and patients with COPD and fibrosis.The segmentation results using the 3D region growing are shown in Figure 6.
A complete thorax CT exam has between among 250 and 1000 images, therefore the results from the 3D region growing method must be evaluated as the evaluation of every single image by a pulmonologist is unvia ble.
The sampling used in this work is based on the human lung, and is divided into four distinct regions (Figure 7) such as 1, 2, 3 and 4 corresponding to the superior lobes, the hilar region, the base and the diaphragmatic region, respectively.
The group of images for analysis is constructed using 3 CT images of each region, totaling 12 images per exam.Since 11 complete exams were used, there were 132 images.These images were used in the evaluation, by the medical experts, of the CT exam lung segmentations, which means that 264 lungs have to be evaluated.This analysis is realized using qualitative metrics due to the impossibility of using quantitative metrics.A quantitative analysis would be unviable as the manual segmentation of the images by a specialist physician is extremely slow and tiring, considering a total of 132 images.
Thus, the qualitative evaluation used in the segmentation of the lungs by the 3D region growing method consists in attributing grades to the segmentation obtained by the method.In this work, two pulmonologists were responsible for this evaluation.The attributed grades in this evaluation possessed values from 1 to 5, in which the respective Qualitative Evaluation Criteria (QEC) of the segmentation is from worst, bad, reasonable, acceptable to optimal.The results are considered satisfactory when they are optimal or acceptable, which represents little to no error in the results.
The physicians who evaluated the results of the methods in question are professors and pulmonologists at the Walter Cantídio Hospital, and were nominated as physician 1 (P1) and physician 2 (P2).The physicians' evaluations, for each classification performance, were presented in Table 3.
The Kappa index value, obtained in the inter observer concordance is 0.684, indicating substantial concordance between the physicians concerning the 3D region growing algorithm.
The indexes obtained in Table 5 show the quality of the segmentation results, and the physicians 1 and 2 considered 64.78% and 68.18%, respectively, of the results satisfactory, having little or no errors.
However the two physicians (1 and 2) identified serious errors in the results obtained by the 3D region growing method in 29.16% and 28.55%, respectively, and these were classified in group 3 of Table 5. Bad segmentations, classified as class 2, were identified by the two physicians in 6.06% and 2.27% of the results obtained by the 3D region growing method.Some examples of the results obtained by the evaluated method are presented in Figures 8 and 9.These figures present the main result types obtained from the physicians' evaluation.Figure 8 presents an illustration of an evaluation type 5 (Table 1), for both lungs according to both physicians.
The most frequent error present in the segmentation of the CT images that the physicians disagreed upon is shown in Figure 9, where the presence of a lung disease, in this case, fibrosis stands out.

Applications for Medical Lung Diagnosis aid
Among the research areas focusing on medical diagnosis aid, the pulmonology area stands out, as various diseases can be detected and tracked using pulmonary CT images.This is possible due to the development of software created for this purpose, in which the lungs present in thorax CT images are extracted and, afterwards, analyzed through Pattern Recognition and Artificial Intelligence algorithms for the detection of diseases and the internal structures of the lung.

3D Reconstruction Visualization by Cuts
The 3D reconstruction of the lung obtained by a complete thorax CT exam enables the visualization of the lung anatomy in 3D.Auxiliary specific cut planes can be inserted in the 3D visualization to aid the specialized physician.This method of visualization through a model and its cut planes increases the accuracy of the physician's diagnosis, as the amount of information for the diagnosis is increased.
To exemplify these planes, Figure 10 presents a 3D model with the axial, sagittal and coronal planes aiding the visualization of the lungs in thorax CT images.It should be noted that this 3D model was obtained through the reconstruction of a CT exam only using axial imagery.The other cut planes are obtained through the reconstructed 3D model.

Analysis and Detection of Airways
Airways are characterized by their hyper aerated bracket and their specific anatomy.They can be detected using the 3D region growing method, and the seed is the most elevated voxel in the lung segmentation and the ruling for neighborhood aggregation is to be in the hyper aerated region.

Analysis and Detection of Emphysema
An example of a disease that can be detected by this technique is pulmonary emphysema.This disease is characterized with a density between −950 and −1000 Hounsfield Units, and thus is considered as a hyper aerated region.This technique can be applied to only one image or to a complete thorax CT exam. Figure 12 presents an image of an emphysema in 3D using the multi-thresholding method.

Detection of Pulmonary Vessels
The blood vessels inside the lung are characterized by their radiological density of between −100 and −500 Hounsfield Units, which is considered a poorly aerated region.An example of the detection of these inside the lung is shown in Figure 13, in which the vessels can be presented individually or inside the lungs.

Bone Region Detection and Analysis
The bone region is characterized as being in the 600 to 2000 HU bracket, and can therefore be detected through a threshold using the cited band.Figure 14 presents an example of the result of multi-thresholding applied to the bone region.

CONCLUSIONS, CONTRIBUTIONS AND FUTURE WORK
The general aim of this work, which was to show the segmentation, reconstruction and visualization of lungs in thorax CT images using the 3D region growing method, was achieved.The results obtained using healthy patients and patients with fibrosis and COPD were analyzed together by   two pulmonologists, and 45.83% and 68.18% of satisfactory results for physicians P1 and P2 were obtained, respectively.The concordance between the two physicians by the Kappa index showed a moderate agreement.
A future work will aim to improve the results of the patients with COPD and fibrosis, as the segmentations of healthy patients achieved good results, which were not the cases with some of the other patients.These less good results are because the diseases alter the radiological density brackets in HU.One of the ways to correct this will be to insert Artificial Intelligence in the clustering of the vicinity neighbors.

Figure 1 .
Figure 1.CT lung image in the axial position: a) multi-detector CT scanner, and b) representation of the density ranges.

Figure 2 .
Figure 2. Illustration of the region growing method: a) original image (seed represented by red color), and b) result of the segmentation (red color).

Figure 3 .
Figure 3. Illustration of the region growing method: a) voxel determination (seed in red color), b) first iteration analyzing neighborhood of the seed voxel, and c) result of the segmentation.

Figure 4 .
Figure 4. Block diagram to apply 3D region growing.

Figure 5 .
Figure 5. Illustration of the 3D region growing evolution (blue color) for the lung segmentation from CT of the thorax: initialization, evolution of the 3D region growing, and result of the segmentation.

Figure 6 .Figure 7 .
Figure 6.Illustration of the lung segmentation from CT exam using 3D region growing with different thickness.

Figure 8 .Figure 9 .
Figure 8. Illustration of the lung segmentation from CT image analyzed by two specialists as optimum (Note 5-Table 1): a) and c) original images, and b) and d) results using 3D region growing.

Figure 11 (
a) shows an example of segmentation of the airway and Figure 11(b) shows the lung segmented without the airways.

Figure 12 .
Figure 12.Full exam showing 3D detection model of a hyperinflated region.

Figure 13 .
Figure 13.3D visualization of the blood vessels.

Figure 14 .
Figure 14.3D reconstruction: a) bone region and b) lung and bone region.

Table 1 .
Evaluative criteria for qualitative image segmentation.

Table 2 .
Illustration of a confusion matrix

Table 4 .
Description of full exam used to evaluate the 3D methods.

Table 5 .
Qualitative evaluation of the specialist P1 and P2.