Automatic Facial Spots and Acnes Detection System

Recently medical cosmetic has attracted significant business opportunity. Micro cosmetic surgery usually involves invasive cosmetic procedures such as non-ablative laser procedure for skin rejuvenation. However, to select an appropriate treatment for skin relies on accurate preoperative evaluations. In this paper, an automatic facial skin defects detection and recognition method is proposed. The system first locates the facial region from the input image. Then, the shapes of faces were recognized using a contour descriptor. The facial features are extracted to define regions of interest and an image segment method is used to extract potential defect. A support-vector-machine-based classifier is then used to classify the potential defects into spots, acnes and normal skin. Experimental results demonstrate effectiveness of the proposed method.


Introduction
Recently medical cosmetology has a great development.It usually involves invasive cosmetic procedures or operation such as non-ablative laser procedure for skin rejuvenation and filling injection to relieve wrinkles [1,2].Most conventional skin analysis instruments are contactbased.A physician has to visually inspect the target region and applies a contact test probe to magnify the target region for inspection.This contact testing procedure may be unsanitary.Non-contact inspections analyze patients' facial skin conditions by a camera directly.Doctors and patients are not necessary to be face to face.Furthermore, remote defect detection for consultation before cosmetology is possible.
VISIA is a widely used commercial instrument for skin analysis in cosmetic surgery.The skin condition is analyzed by multispectral images [3].However, the cost is high and physicians should manual outline regions of interest (ROIs) during inspection.
With gradually higher resolution in digital cameras, many digital imaging methods have been proposed to analyze skin conditions [4][5][6][7].These investigations applied various color quantization methods to distinguish whether the ROI is a spot or not.However, using pure color information to detect spots is difficult because shadow of facial organs (eyes, nose, mouth, or ears) may be misjudged as spots.
In addition, most skin analysis systems require manually outlining the regions of interest (ROIs) [3][4][5].How-ever, manual ROI outlining is commonly known as a time-consuming and non-repeatable process.
In our prior work, an automatic facial skin defect detection system was proposed [7].However, the facial view was restricted to front.Hence, a novel skin conditions evaluation system, which integrates a multi-view image acquisition device and automatic facial skin defect detection, is proposed in this paper.A facial features detection approach is applied to obtain positions of the facial features which are further applied to extract the ROI.A special color space and an adaptive threshold based on probability distribution are used to characterize potential defects from the ROI.Finally some significant texture features are extracted and applied to a specific designed classifier to classify potential defects into normal skin, spots and acnes.
The rest of this paper is organized as follows.Section 2 introduces details of the proposed skin defects detection and recognition system including definition of ROI, potential defect extraction and texture features used for classification.Section 3 demonstrates the experimental result.Conclusions and possible research topics in the future are given in Section 4.

The Proposed System
In this paper, an automatic facial skin defects detection and recognition system is proposed.When the facial images are captured by a high definition camera, the skin defects are detected and recognized automatically.

Facial Region Detection
A skin color detection method is applied to detect facial region.Soriano [8] proposed a skin locus in normalized color coordinate (NCC).This skin color model has performed well with images under widely varying conditions [9].Therefore, the skin locus is adopted to detect skin-like region in this paper.A region filling method is applied to the largest connected region which is obtained by an 8-adjacent connected component labeling algorithm.The largest skin region is regarded as the facial region [10].The facial region detection result is shown in the Figure 2.

Facial View Detection
Before extraction of the ROI, the facial view needs to be determined.The shape of face can be used to distinguish between front and profile.Fourier descriptors have been widely used for representing boundary of a two-dimensional shape and it has the advantages of affine-invariant such as translation, rotation, and scaling [10].In these descriptors, the centroid distance is used to represent the shape.The centroid distance Cd(t) is defined as follow: where (x t , y t ) is the t-th point on the face contour, (x c , y c ) is the centroid of the face contour, and N is the number of sampling points.Fourier transform of Cd(t) is defined as follow: Since Cd(t) is a real value, there are only N/2 different frequencies in the Fourier transform, only half of FD k is needed to represent the face contour.Scale invariance is then obtained by dividing the magnitude values of the first half of FD k by DC component.The feature vector F is used as the input vector of support vector machine for training and testing.
Because the Fourier descriptors have the attribution of affine-invariant, left and right profiles have the same result from the classifier.Hence, the direction of profile needs to be detected.The skin color distribution is used to distinguish between left and right profile.The numbers of skin pixels in the left profile and right profile are calculated respectively.
Figure 3 shows the statistics of skin pixels in the left profile and right profile, where R and L represent the number of skin pixel in the right profile and the left one, respectively.If L is large than R, the direction of face is right, otherwise is left.

ROI Extraction
The facial features are further adopted to exclude undesired regions, including eyes, eyebrows, mouth and nostrils.The facial features are detected beforehand using empiricals information in YCbCr and HSV color spaces [7,11].First, the pupils are detected under the YCbCr color space in the rough regions of both eyes.The rough

Potential Defect Extraction
In order to extract potential defects from the ROI, a segmentation method for potential defect detection is proposed.Because the surface of face is not a flat surface,   skin color may varied in different positions on the same objects, even though the area focuses only on the forehead [7].Therefore, the ROI image is divided into n nonoverlap sub-images regularly.In this paper, the width and height of every sub-image are set to 100.The selection of color space is important for pattern recognition.Cr in YCbCr and AngleA has well human visual feedback for acnes and spots, respectively [12].Therefore, Cr-AngleA color space is adopted to detect potential defects.The transformation functions are defined as follows: where and R, G, B are three values corresponding to red, green and blue component in the original RGB color space.The range of Cr value is [0, 255] and AngleA is [0, 90]. Figure 6 shows an example of skin and defects distribution in Cr-AngleA space between two sub-images.We can find that the spot and acne have relatively high values in AngleA and Cr, respectively.
The center of skin color distribution (Cx, Cy) is obtained by the maximum count of histogram in Cr and AngleA respectively.Cx and Cy are defined as follows: where H k () is the histogram in the k level.The distance transform D(x, y) between observational value and the center is defined as follow: The curve fitting function f k (λ) which is shift by the Poisson is defined as follows: where Then, f k (λ) is scale to match p k , Equation ( 12) is modified to Equation ( 14) which is defined as: where λ = 0, 1, , L. The fittest curve p* k is defined by finding a λ such that the difference between p k and f' k (λ) is minimum, and it is defined as follows: * argmin ( ) The threshold t is set to the maximum value of the second order derivatives of p* k .In other words, the pixel is regarded as foreground if the distance between observational pixel and the center in Cr-AngleA color space is larger than t; otherwise it is regarded as background.The threshold t is defined as Equation ( 16).

 
where () is the second order derivative.The fittest curve and its second order derivative are shown in the Figure 7.
The potential defect is denoted by S(x, y) Figure 8 shows the potential defect extraction method step by step.The potential defects can be obtained by a 4-adjacent connected component labeling algorithm.

Feature Extraction
To further classify the potential defects into normal patterns, acnes, and spots, the texture features calculated from co-occurrence matrix are used.The co-occurrence matrix is introduced by Haralick [13], which indicates probability of grey-level i occurring in the neighborhood of grey-level j at a distance d and direction .Fourteen co-occurrence matrix features including Contrast, Homogeneity, Mean, Variance, Energy, Entropy, Angular Second Moment, Correlation, Sum Average, Sum Entropy, Difference Average, Difference Variance and Difference Entropy are extracted [14].In this paper, the distance was chosen as one pixel and four angles (0˚, 45˚, 90˚, 135˚) were selected.
To preserve spatial details, the texture features of the four directions are averaged.In addition to gray color, the color space Cr and AngleA are also adopted.Accordingly, there are 14 × 3 GLCM features calculated from each potential defect.The averaged GLCM feature in the color space C is denoted as ,0 ,45 ,90 ,135 4 where , C f t  represents the feature f with angle  in color C. A geometric feature roundness defined as Equation ( 17) is also adopted.
where E and A are the perimeter and the area in a poten-  tial defect, respectively.Therefore, there are 42 + 1 features in candidate features.

Feature Selection
In order to sift the most significant features, the sequential floating forward selection (SFFS) is adopted for feature selection [15].This method consists of applying sequential forward selection (SFS) and sequential backward selection (SBS).By using the SFS and SBS repeatedly, it will converge when the defect recognition rate does not increase.In this paper, the cross validation rate is defined as the defect recognition rate which is implemented by LIBSVM [16].There are 126 acnes, 134 spots and 134 normal patterns which are extracted manually from the database.Figure 9 shows some examples of the training patterns.

Feature Extraction
Support vector machine (SVM) is a popular and robust classifier in classification and regression analysis task.
The SVM constructs a hyperplane in a high dimensional space which has the largest distance to the nearest training data points of any class.Figure 10 shows the struc-ture of the classifier.The classifier is utilized to classify the potential skin defects into normal patterns, acnes and spots, and the structure of classifier is a decision tree structure which consists of two SVMs.The SVM1 is used to classify defects and normal patterns from the potential defects.The other SVM2 is used to classify acnes and spots from the defects.Moreover, the SFFS algorithm is carried out individually in each stage.

Experiment Environment
To reduce the influence of illumination, a special designed image acquisition device is created.Faces are captured by a high-resolution camera with resolution of 10 M pixels.
Figure 11 shows the acquisition device.In this device, the acquisition parameters including camera parameters, light source, distances from subjects' face to the camera are all fixed.The camera is Cannon Power Shot G10.ISO speed is set to 1/60 sec.Focal length is set to automatic mode.Because the forehead is an important part in medical cosmetology, every subject is asked to move   their hair aside in order to show their forehead.Figure 12 shows some samples captured in this study.The system is implemented with Visual C# and Python 2.5 on an Intel Core 2 Quad 2.66 GHz processor and 2 GB RAM platform.

Experimental Database
To demonstrate the capability of the proposed method, there are two face image database YUFH and YUFH2 are used.YUFH was created by Chang [7], and there are 93 face images taken from 3 females and 26 males in 3 views (front, left profile and right profile).YUFH2 is collected by our image acquisition device.There are 54 face images taken from 3 females and 15 males in 3 views.

Face View Classification Result
To demonstrate the capability of the face view classification, the two databases are used.In each database, a half of subjects are used for training, rest of subjects are used for testing.The Confusion Matrix for face view classification result in the database YUFH and YUFH2 are shown in Tables 1 and 2 respectively.

Feature Selection Result
The result of feature selection by SFFS for SVM1 is shown  in Table 3.For SVM2, the result of feature selection by SFFS is shown in Table 4.

Defect Detection Result
To quantify the performance of the proposed approach, three standardized measurements are adopted: Accuracy, Sensitivity and Specificity.They are defined respectively as follows: Accuracy Sensitivity where R P is the total number of defect regions and R N is the total number of normal regions.R TP is the number of regions in the actual defect region and is classified as defect by the proposed system and R TN is the number of regions in the actual normal region and classified as normal by the system.In this paper, the R N is defined as R P subtracted from the total number of potential defect regions.
Table 5 shows the evaluation results of the proposed approach in the YUFH database.There are 93 face im-  ages including front and profile.Table 6 shows the evaluation results of the proposed approach in the YUFH2 database.There are 54 face images including front and profile.In these tables, the Sensitivity measurement in acne detection is relatively low than the others.The reason is that acnes have various types and stages in clinical treatment.
Figure 13 shows the result of defects detection, in which the spots and acnes are outlined in blue and red, respectively.
We compared the performance of the proposed approach for spots with Chang [6] and Chang [7].shows the results obtained using the proposed approach and [6,7].There are 97 images including 32 males and 4 females from both YUFH and YUFH2.From this table, one can see accuracies of the proposed approach for spots detection are higher than those of [6] and [7].

Conclusions and Future Directions
Skin analysis is one of the most important procedures before medical cosmetology.In this paper, an automatic facial skin defects detection and recognition system is proposed.Different from other methods which have to manually outline the ROIs in a face image, the proposed approach obtains the ROI automatically in both front and profile views.
The proposed system first locates the facial region from the input image.The facial features and skin color were used to locate the ROI.An approximate Poisson distribution is used to define a suitable threshold for extracting potential defects.Then, the SFFS is adopted to select significant features for defects classification.Finally, a decision tree classifier consists of two SVMs is applied to classify the potential defects into normal patterns, spots and acnes.Experimental results show that the proposed method can detect facial skin defect and recognize the lesion effectively.
In the future work, more images of subjects in different ages will be collected and tested.The accuracy of the current system can increase by attempting more features.

Fig- ure 1
illustrates the processes of the proposed approach, includes face region detection, facial view detection, front/profile ROI extraction, potential defect extraction and classification.Details of these processes are described in the following subsections.

Figure 1 .
Figure 1.The flowchart of the proposed system.

Figure 3 . 4 .
Figure 3. (a) Right profile; (b) Left profile; (c) The number of skin pixel in the right profile; (d) The number of skin pixel in the left profile.regions of other facial features are located according to the position of pupils, as shown in the Figure 4.The eyes, eyebrows, mouth and nostrils are extracted by the Sobel, HSV color space, YCbCr color space and difference of Gaussian respectively in the corresponding region.The ROI is obtained by removing the facial features from the facial region.Figure 5 shows the results of ROI extraction in different view.

Figure 5
shows the results of ROI extraction in different view.

Figure 5 .
Figure 5.The results of ROI extraction.

Figure 6 . 2   ( 10 )
Figure 6.The example of skin and defects distribution in Cr-AngleA space.

Figure 7 .
Figure 7.The result of threshold detection in a sub-image.

Figure 8 .
Figure 8.The potential defects extraction processing.

Figure 10 .
Figure 10.The structure of classifier.