Ultrasonographic Segmentation of Fetal Lung with Deep Learning

The morbidity and mortality of the fetus is related closely with the neonatal respiratory morbidity, which was caused by the immaturity of the fetal lung primarily. The amniocentesis has been used in clinics to evaluate the maturity of the fetal lung, which is invasive, expensive and time-consuming. Ultrasonography has been developed to examine the fetal lung quantitatively in the past decades as a non-invasive method. However, the contour of the fetal lung required by existing studies was delineated in manual. An automated segmentation approach could not only improve the objectiveness of those studies, but also offer a quantitative way to monitor the development of the fetal lung in terms of morphological parameters based on the segmentation. In view of this, we proposed a deep learning model for automated fetal lung segmentation and measurement. The model was constructed based on the U-Net. It was trained by 3500 data sets augmented from 250 ultrasound images with both the fetal lung and heart manually delineated, and then tested on 50 ultrasound data sets. With the proposed method, the fetal lung and cardiac area were automatically segmented with the accuracy, average IoU, sensitivity and precision being 0.98, 0.79, 0.881 and 0.886, respectively.

ing technology for evaluating the lung immaturity due to its unique nature being radiation-free and cost-effective.
Recently, computer-aided technologies were applied in the fetal lung assessment which was considered as quantitative and objective. In view of this, Tekesin et al. [6] evaluated the fetal lung development through quantitative ultrasonic tissue characterization. In their study, the histogram of gray values within a manually delineated region of interest (ROI) was analyzed. Researchers in Spain developed a semi-automatic quantitative ultrasound analysis method for FLM evaluation by analyzing the texture information of the fetal lung using ultrasound, in which the regions of the fetal lungs were manually delineated [7] [8].
However, most studies were semi-automatic because the fetal lung region in an ultrasound image was delineated manually by the clinicians. This may cause inter-operator errors, and introduce bias to the final evaluation. Thus, an automatic method for the fetal lung segmentation may not only minimize the inter-operator errors and improve the effectiveness of ultrasonographic fetal lung assessment, but also offer an objective method for the evaluation of the fetal lung development based on the morphological characteristics of the fetal lung.
On the other hand, with the development of the deep learning, image segmentation has been one of the hot topics in the community of image processing in the past decades. In view of this, Havaei et al. [9] presented a fully automatic brain tumor segmentation method based on deep neural networks. Hu et al. [10] proposed a method for computed tomography (CT) lung segmentation using mask region convolutional neural networks combined with supervised and unsupervised machine learning methods. However, rare researchers attempt on the segmentation of the fetal lung and heart on ultrasound images using deep learning.
Considered above, we proposed a deep learning method for automated fetal lung segmentation on ultrasound images using U-Net in this study. with more than 6-year experience in the field of fetal ultrasound delineated the regions of the fetal lung and heart using a web-based tool for image annotation (Labelme) [11], as shown in Figure 1. The manually delineated fetal lung and heart was subsequently served as the ground truth for evaluating the automated segmentation method. The study was approved by the ethics committee of Nanjing Medical University Affiliated Suzhou Hospital, Suzhou, China.

Image Pre-Processing and Data Augmentation
All the ultrasound images used in the study were cropped with the image part in one of the three RGB channels remain for the subsequent process. Data augmentation was applied to the training data set via rotation, flip, and shift transformation, which is broadly used in deep learning [12], to increase the number of training data and improve the robustness of the model. Such an operation enlarged the training data set to 3500 images.

U-Net Model
In healthcare, large medical data are far from available, since the lack of medical data and experienced clinicians to annotate images. In view of this, the model should be well designed to avoid overfitting. In this study, the U-Net network was applied recognizing its good performance in the segmentation of medical images [13].
The U-Net architecture is based on the full convolutional network (FCN) [14].
Compared to FCN, the U-Net performed up-sampling for four times and used skip connection in the same stage instead of directly supervising and loss-reverting on high-level semantic features [13]. In this way, it ensures that the final recovered feature map not only incorporates both high resolution and high-level semantic information, but also allows fusion of multi-scales features, leading to multi-scale prediction and deep supervision.
The entire U-Net model contains two processes: the down-sampling process   [16]. The U-Net architecture used in this study is shown in Figure 1.

Training and Implementation
The original data set consists of 300 ultrasound images with the fetal lung and heart annotated, and was divided into the training data set (250 images) and testing data set (50 images). The data augmentation was performed on the training data set via rotation, flip and shift transformation, resulting in a 14-time-enlarged data set containing 3500 images. Noted that the testing and training samples were independent without overlaps.
The model was trained on a computer with a single GPU and 12 GB memory with the Adam optimizer [17]. In addition, the initial learning rate, maximum epochs, mini-batch size were set to be 0.001, 100 and 2 respectively. Considering the low computing resources, the batch size is set as 2 to avoid system collapse in this study.

Evaluation Metrics
To evaluate the performance of the proposed method, the predicted segmentation results was compared with the manual annotated labels in terms of accura-

Statistical Analysis
Bland-Altman test was used to test the agreement between the automated and manual measurements for the segmented fetal lung and heart. SPSS (version 22.0

Results
The performance of the U-Net model was calculated in terms of accuracy, precision, IOU and recall, as listed in Table 2. Figure 2 shows three segmented results. The Bland-Altman analysis was shown in Table 3 and Table 4 for evaluating

Discussion
The segmentation of the fetal lung from ultrasound images is one of the challenging and crucial steps for the computer-aided evaluation of the FLM. In this study, we proposed an automated segmentation technique for segmenting the fetal lung in ultrasound images.
By training on 3500 annotated ultrasound images with 50 epochs, the proposed models showed good performance in segmenting the fetal lung in terms of accuracy, precision, IOU and recall. Figure 2 shows the segmentation results using the U-Net trained with fetal lung and cardiac region annotations. It shows that lung area can be accurately segmented compared with the manual annotations. In addition, the Bland-Altman tests as shown in Table 3 and Table 4 demonstrated the agreement of the manual and automated methods for segmenting fetal lung and heart.
Segmentation of the fetal lung would not only support the computer-aided fetal lung evaluations, e.g., the texture analysis for the prediction of FLM, but also assist in the monitoring of the fetal development. In this regard, previous work adopted the segmentation of the fetal brain and tissue to assess the fetal development [18] [19].
This study presented some limitations. Firstly, the regions of the fetal lung in the ultrasound images of the training and testing data sets were delineated by physicians, which may introduce bias. Secondly, the data used in this study were only collected from Chinese fetuses and a limited number of ultrasound machines.
In future, the performance of our proposed method on fetal lung images from other countries would be evaluated, and other feature extractors would be tried to improve the model considering the above limitations.

Conclusion
This study proposed a robust method for automatic fetal lung segmentation in ultrasound images using U-Net model. By training on 3500 ultrasound fetal lung images, the proposed model could segment the fetal lung with a good accuracy.
The proposed model could be potentially applied not only to improve existing studies in quantitative analyzing the fetal lung using ultrasound, e.g., texture analysis of fetal lung and prediction of the neonatal respiratory morbidity, but also to assist the clinicians in daily measurement of the fetal lung/heart.

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.