A Local Binary Pattern-Based Method for Color and Multicomponent Texture Analysis ()
1. Introduction
Texture is an important image investigation approach efficient for content analysis and feature extraction. It describes the spatial arrangement between image pixels. Its analysis in content-based images analysis is used in many research domains including medical image analysis [1], content-based image retrieval [2], remote sensing imagery [3], object recognition [4], object classification [5]. Yet, no proper definition has been given to texture, but words like coarseness, regularity, fineness, smoothness, granulation, randomness, contrast and lineation are qualitatively used to evaluate it [6].
Images content analysis based on texture is a complex task to achieve. An investigation [7] [8] [9] on it has led to different approaches such as statistical approaches [10] [11], structural approaches [12] [13], transform-based approaches [14] [15], model-based approaches [16] [17], graph-based approaches [18] [19], learning-based approaches [20] [21] and entropy-based approaches [22] [23]. Each category is suitable for a type of images depending on their nature, how they have been acquired or the goal the user wants to achieve. Here we are interested in texture classification which is an application of texture analysis aiming to label images basing on their properties.
Texture classification can be divided into two steps. The first one concerns the feature extraction while the second one leads to classifier designation. The use of powerful feature with a simple classifier can lead to a good texture classification but the inverse is not right [24] [25]. When a method for texture description is being designed, two challenges must be taken into account: high quality feature and low computational complexity [24]. On one side, a high-quality feature is supposed to be distinctiveness and also robustness. The distinctiveness is the capability of the extracted feature to accurately represent different types of textures while robust features tend at describing textures resulting from images acquisition system imperfection. These include rotation changes, illumination variation, presence of noise and occlusion. On another side, computation of the texture feature must be low and fast for application running in real-time: these have been the main points of the success of Local Binary Patterns as texture descriptor.
Recently, Local Binary Patterns (LBPs) from statistical approach based-texture analysis, initially known as Texture Unit [26], latter improved in [27], received more attention in the field of computer vision and image analysis for their low computational complexity, invariance to monotonic illumination, ease of implementation, robustness in gray scale variations [25] [27]. Its principle is in a circular neighborhood consisted of P pixels’ intensity defined on a Radius R, the neighbor pixels intensity are thresholded against the central pixel intensity then weighed by a power of two. Then, a histogram is used to model the distribution of LBPs codes.
Initially, LBPs-based methods have been introduced for gray scale images. Some research have been done to extend LBP operator to multicomponent images analysis. Those methods, depending on how they analyze the spectral information, can be roughly divided into three groups. The first one, easy to implement, consists in computing LBP operator on each image component. Then histograms from each component after applying LBP operator are concatenated and used as texture feature. Known as marginal approach, this technique undergoes the limitation to not consider correlation existing between spectral components [28] [29] [30]. The second group was carried out for color images. Because they are based on color information, they are not suitable to multicomponent images out of the range (400 nm to 700 nm) [31] - [36]. Finally, the pioneer LBP method [37] which is the cornerstone of the third group aims at analyzing multicomponent images per pair of spectral components. Because of its feature dimensionality increasing with the number of image components, many studies have been conducted to reduce it [38] [39] [40] [41] [42]. The limitation of this operator is to ignore the full correlation between image components by considering two spectral components at the same time. In the next, we propose a new formulation of LBP principle capable of fully taking into account the correlation between spectral components with a feature dimensionality independent on the number of the spectral components.
2. Material and Methods
2.1. Material
Outex databases [27] [43]: Outex framework consists of a large set of images textures acquired under different conditions for empirical evaluation of algorithms designed for texture analysis. Images are captured using a Macbeth Spectra Light II Lumin are light source coupled to a Sony DXC-755P three chip CCD camera attached to a GMFamuc S-10. The illumination sources are controlled by a workstation to select the desired illuminant among 2300 K horizon sunlight denoted “horizon”, 2856 K incandescent CIE A denoted “inca” or 4000K fluorescent TL084 denoted “tl084”. The camera can be rotated into a desired angle by a robot arm to produce images at different rotation angle. From Outex databases, we selected Outex 10-C for rotation invariant test, Outex 33 for blurred images test, Outex 31 for illumination variation test.
Outex 10-C is a set of 24 textures classes. Each class consists of 20 images acquired at different rotation angles: 0˚, 05˚, 10˚, 30˚, 45˚, 60˚, 75˚ and 90˚. From these data, images acquired at 0˚ rotation angle were used as training data while the images of other angles were used as testing data. Consequently, 480 (20 * 24) images were used as training data and 3840 (8 * 20 * 24) images as testing data. From Outex 33 consisted of 68 textures classes and 20 images per texture class, 1360 (20 * 68) images were used as training data and the same data with Gaussian blur added to them were used as testing data. Outex 31 is similar to Outex 33 but images were acquired with illumination variations. The training data were acquired using 2856 K incandescent as illuminant while the test data were obtained with the illuminant 2300 K horizon. The Outex databases are available on http://www.outex.oulu.fi.
Multispectral database [44] [45] [46]: multispectral images used in our work were acquired from human red blood cells. Samples are collected from fingers of malaria-infected patients. Next, there are spread on corrosion-resistant glasses and dried for 7 - 15 minutes each one. The samples are not stained to preserve the optical properties of the blood cells. Then, they are imaged using a multispectral and multimodal microscope. The used microscope is a modified Brunel Compound microscope designed to enable images acquisition with more than three spectral bands. Equipped with 13 LEDs ranging from 370 nm to 940 nm, it has three imaging modalities (Transmission, Reflection and Scattering). On the same scene of the sample and for each LED, an image is captured using a 12-bit pixel depth monochrome CMOS camera of 5MP (2592 × 1944, Guppy-503B, Allied Vision Technology constituted of a MT9P031 sensor from Micron/Alpha). To remove noise from the acquisition environment and the electronic components, in each modality, images from the sample (IS), the reference (IR) and the dark (ID) are captured and preprocess as below to get the final spectral image:
(1)
Finally, an image consisted of 13 spectral bands is obtained in each modality. In our experimentation, only images from Transmission modality were used. In our experimentation, 500 red blood cells images were used as training data while 1800 as testing data. The data are from 10 different samples.
2.2. Methods
2.2.1. A Brief Review of LBP Operator
The principle of LBP operator applied to a gray level image is to encode the pixels value in a region along a circle consisting of P pixels intensity on a radius R as follows [27]:
(2)
With S an operator retaining the sign of differences defined by:
(3)
gi denotes the pixels intensity in the defined neighborhood and gc the central pixel intensity. The determination of the position of gi in the image depends on the topology of the neighborhood. The most common used topology is circular so that the coordinates of the neighbors [27] [47] are obtained by:
(4)
With gcx and gcy denoting the x and y coordinates of the central pixel in the neighborhood. All values that don’t exactly fail on a pixel intensity in the image are approximated by bilinear interpolation. Then histogram is used to model the distribution of LBP codes as bellows:
(5)
With
and:
(6)
M and N represent the number of LBP codes along x and y axis respectively. To improve the distinctiveness of LBP operator [27], the LBP codes histogram is built so that different labels are given to certain special codes LBP called “uniform pattern” while the “non-uniform” ones are assigned to the same label. Uniformity is a measure denoting the number of binary changes in a LBP code. Consequently, a LBP code is labeled as uniform if the uniformity measure defined below is at most 2:
(7)
mod is the modulo operator, gi are the pixels intensity in the defined neighborhood, gc the central pixel intensity and S the operator defined in Equation (3). For a given number P of pixels values in a neighborhood, the maximum number of uniform patterns is equal to
ranging from 0 to
. The application of above uniformity measure to LBPR,P operator leads to a better operator for texture classification [27]. The new LBP operator following the uniformity measure idea is formulated as follows:
(8)
Equation (8) aims at grouping LBPs codes having at most 2 transitions under different labels. However, the ones with more than 2 transitions are on the same label. This process is realized during the histogram construction. In LBP concept, the transition is the bit change in LBP codes (from 0 to 1 or from 1 to 0). To build a rotation invariant operator, LBP operator has been modified to retrieve the number of bit changes in LBPs codes. This gives:
(9)
The number of
codes is P + 2 ranging from 0 to P + 1.
2.2.2. Our Proposed Approach
The LBP operator designed in [27] locally describes the spatial structure of an image by encoding the differences between neighbor pixels intensity and central pixel intensity: those differences are encoded depending of their signs. Extend this principle to vectors for analyzing images having more than one spectral band requires to define a way of computing differences between vectors. To follow the idea of LBP operator based of encoding differences that can be negative or positive, we propose the below formulation:
(10)
With
defined by:
(11)
ST is an operator whose mathematical formula is:
(12)
is the qth component of ith neighbor vector while
is the qth component of the central vector in a circular neighborhood consisting of P vectors located on a radius R. Z represents the number of components that consists a vector which is equal to the number of spectral components and T is a threshold value.
in Equation (11) is similar to (gi − gc) in Equation (2), thereby it can be positive or negative. LBP operator defined in Equation (2) has many drawbacks like its incapacity to capture macrotexture, its sensitivity to noise and its lack of robustness. To better address these disadvantages, many LBPs variants have been worked out [24] [47] [48] [49]. From all works on LBP improvement, it can be roughly established that the use of a median threshold is better than a null one [48] and the combination of LBPs methods capable of capturing micro-texture with the ones capturing macrotextures is better than single LBP method capturing either microtexture or macrotexture [24] [49]. Based, on those conclusions we define T so that the final operator is capable of capturing both micro and macro textures. For micro-textures, T is defined as follows:
(13)
where
is the absolute value of x and median the median value of a set of values. Its mathematical formula [50] is:
(14)
where argmin is the operator retaining the minimum value of a set and M the number of scalar values X in the set. For macro-textures, T = T2 is defined to be the median of the absolute value of the image after applying the Equation (11) on all image vectors. The use of T = T1 in the Equation (10) produces a new operator termed MLBP_LR,P with L standing for local while the use of T = T2 produces a new one MLBP_GR,P with G meaning Global. Also, the principles of Equations (8) and (9) can be apply to MLBP_LR,P and MLBP_GR,P to improve their distinctiveness in textures classification. Finally, the histograms from MLBP_LR,P and MLBP_GR,P can be used separately as texture feature, but to capture both micro and macro structures their joint histogram is used as final texture feature.
3. Results and Discussion
This part of our work concerns testing the performance of our approach with respect to the main techniques proposed in the literature. A supervised textures classification can be divided into two parts: the feature designing that has been previously presented and the use of a classifier.
3.1. Choice of A Classifier
Our goal is not to design a classifier but to test the performance of methods proposed to analyze images texture. Thereby, we select the simple Nearest Neighbor Classifier (NNC) utilizing one neighbor with the X2 distance metric [24] [39] [47] [48] [49]. The X2 distance metric between two distributions is given by:
(15)
where B is the number of bins in the histogram, S and M are the test and model samples respectively. Sb and Mb represent the bth bin of two different histograms, so having Sb + Mb = 0 means Sb − Mb = 0; thereby the algorithm is built so that
the ratio
is null when Sb + Mb = 0.
3.2. Methods in Comparison
The evaluation of our approach was done in comparison with some methods dealing with multispectral images. We first implement the simplest multispectral LBP operator used in [28] [29] [30]. It is computed on each image component, then histograms are concatenated as follows:
(16)
With n denoting the nth image component. We second implement the LBP method designed in [37] whose mathematical formula is given below:
(17)
where b2 and b1 denote the spectral band of the image, gi the pixels intensity in the defined neighborhood, gc the central pixel intensity, S the operator defined in Equation (3) and P the number of points defined in the neighborhood. Then histograms after applying the above equation are concatenated as:
(18)
We third implement multispectral LBP according to the vectors’ norm presented in [51]. We chose it because its principle can be applied to an image having more than three spectral bands without taking into account the color information as:
(19)
The norm of a vector is defined as:
(20)
Also, the results from our proposal MLBP_LR,P, MLBP_GR,P and MLBP_JR,P for the joint histogram of MLBP_LR,P and MLBP_GR,P are presented.
We have tested the implemented LBP-based methods on different schemes of radius R and the number P of pixels value in a neighborhood that are (1, 8), (2, 16) and (3, 24). The use of different schemes is to examine their efficiency in LBP methods-based textures analysis. This leads to select the best resolution by comparing the provided features dimensionality and classification score. We also achieved multiresolution analysis defined in [27] whose principle consists in concatenating LBPs codes obtained using Equation (8) or (9) by changing the values of R and P. This leads to a more powerful texture feature. In this work, different quantization have been used namely:
;
;
;
. The goal of testing LBP based-methods on different quantization of R and P is to check at which resolution LBP operators provide the best classification rates.
3.3. Experimentations Results and Discussion
From experimentations conducted on databases using single scheme of (R, P), results from operators established in (17) and (16) are almost equal (Figures 1-3). Equation (17) combines texture features of equation (16) and the ones obtained by computing LBP operator of each pair of spectral bands. This leads to a redundancy of features affecting the performance of the operator. The main advantage of using marginal approach is its ease of implementation. Our proposed approach combining micro and macro features gives the best scores as shown below. NLBP from (19) consists in computing LBP operator (2) after a transformation of the image. The transformation produces an image which is a combination of the spectral bands. This combination is less accurate with our own estimating the variation between vectors on the basis of the sum of their differences.
Figure 1. Test of LBP-based methods on Outex 10-C using single (R, P).
Figure 2. Test of LBP-based methods on Outex 31 using single (R, P).
Figure 3. Test of LBP-based methods on Outex 33 using single (R, P).
On Figure 1, we can observe a score higher than 95% using our approach on images subject to rotation.
On the database with illumination variation showed on Figure 2, the maximum score is still obtained using our approach but decreases over the scheme (R, P) = (2, 16). Over that scheme, the macro feature (MLBP_G) gives the best performance. These results are due to the fact that the macro feature is obtained by a global threshold of the image taking into account the global variation of the light [24] [49]. We can notice the incapacity of the micro feature to give better score in an illumination variation case, compared to the one from macro feature. These results show the efficiency of macro feature as a powerful tool to texture analysis. The efficiency of macro feature compared to micro feature relies on the global illumination variation in the texture.
On Figure 3 showing the experimentation on blurred images, a 100% score is recorded with our approach from the (2, 16) scheme leading to a powerful operator for this kind of database.
The multiresolution analysis considerably improves the performance of the operators with the disadvantage of increasing the feature dimensionality. From a score higher than 95% obtained with our approach, this number is now 99% using this strategy (Figure 4). We can also see an improvement of others operators.
Different from what we observed on Figure 2, the multiresolution analysis improves the performance of our approach over all schemes combining micro and macro features (Figure 5).
On Figure 6, we clearly see the performance of the multiresolution analysis. These results indicate our method to be efficient for blurred images classification.
Figure 7 and Figure 8 exhibit experimentations conducted on red blood cells images with more than three spectral bands. On Figure 7, we can see MLBP combining MLBP_G and MLBP_L gives the best results classification. The highest score is obtained at (1, 8) scheme.
Figure 4. Test of LBP-based methods on Outex 10-C combining different (R, P).
Figure 5. Test of LBP-based methods on Outex 31 combining different (R, P).
Figure 6. Test of LBP-based methods on Outex 33 combining different (R, P).
Figure 7. Test of LBP-based methods on multicomponent images using single (R, P).
Figure 8 presents multicomponent images classification using different schemes of radius R and the number P of points in the neighborhood. Below, operator analyzing micro structures gives the best results with the increasing of the scheme. From (2, 16) + (4, 16), a decreasing of MLBP is observed. This is due to MLBP_G giving low score compared to MLBP_L. MLBP_L is efficient to classify content where differences are micro observed as in the case of red blood cells.
The big deal in designing an operator to analyze multicomponent texture is how to take into account texture and spectral information. The combination of operator analyzing micro texture with the one analyzing macro texture has proven to be powerful in texture classification.
As previously said, the aim at classifying images using different schemes is to detect the best classification scheme. Using LBP-based methods, the highest is the number of points P, the more the texture features increase leading to slow down the feature computation. In all experimentations, our proposed method at least gives 90% score at (1, 8) scheme.
Figure 8. Test of LBP-based methods on multicomponent images combining different (R, P)
The multiresolution analysis is an interesting approach to texture analysis based on Local Binary Pattern. It leads at combining meaningful textures characteristics. Those meaningful characteristics are obtained by varying the radius and the number of pixels intensities. Its main limit is the feature dimensionality increasing. Also, the increasing of the number of resolutions doesn’t not provide better scores as presented in Figure 8. Conversely, it can produce worse results. This can be explained by the fact that features from some resolutions are not meaningful to the analyzed texture. They don’t efficiently describe the variations existing in the texture. In this case, the addition of those insignificant features destroys the robustness of the final texture feature.
4. Conclusion
In this letter, we address new method based on Local Binary Patterns’ principle for color and multicomponent texture analysis purpose. We found the proposed approach efficient for texture classification. Because no assumption was done, our strategy is suitable for color images and images with more than three spectral bands. Also, the texture feature obtained does not depend on the number of spectral bands.
Acknowledgements
This study was supported by ISP (International Science Programme, UPSALA Sweden University) so thanks are due to it.
Future Work
In this correspondence, we propose a multicomponent texture operator combining micro and macro structures. From above results, when classifying images where differences are macro-observed, the combination of MLBP_L and MLBP_G leads to a more powerful operator. Nevertheless, when they are micro-observed the final operator tends more or less to MLBP_L performance. Consequently, future work can be to design a new operator to determine when to separately use or combine both operators for a more powerful operator.