Shadow Detection Method Based on HMRF with Soft Edges for High-Resolution Remote-Sensing Images

Shadow detection is a crucial task in high-resolution remote-sensing image processing. Various shadow detection methods have been explored during the last decades. These methods did improve the detection accuracy but are still not robust enough to get satisfactory results for failing to extract enough information from the original images. To take full advantage of various features of shadows, a new method combining edges information with the spectral and spatial information is proposed in this paper. As known, edge is one of the most important characteristics in the high-resolution remote-sensing images. Unfortunately, in shadow detection, it is a high-risk strategy to determine whether a pixel is the edge or not strictly because intensity values on shadow boundaries are always between those in shadow and non-shadow areas. Therefore, a soft edge description model is developed to describe the degree of each pixel belonging to the edges or not. Sequentially, the soft edge description is incorporating to a fuzzy clustering procedure based on HMRF (Hidden Markov Random Fields), in which more appropriate spatial contextual information can be used. More concretely, it consists of two components: the soft edge description model and an iterative shadow detection algorithm. Experiments on several remote sensing images have shown that the proposed method can obtain more accurate shadow detection results.

tral features of multiband images, some researches explored invariant color spaces to stress the differences between shadows and non-shadows [4] [5] [6]. Also, some efforts on combining different spectral features [7] [8] have achieved some good results in their intended realms. However, there are still some limits in these methods because of the absence of other information such as edges and spatial context information.
As for spatial information, the well-known probabilistic model HMRF which serve as a powerful formal tool to present neighborhood interactions is a natural choice [9]. In recent years, many researchers have attempted to incorporate more information into it to improve the performance of HMRF [10] [11]. Our latest work [12] adds edge constraints into the iterative clustering procedure based on HMRF. In this work, an edges consistency model is proposed to describe the similarity between the clustering edges and the pre-detected ones. This method can obtain more clear boundaries along with the homogeneous area.
However, it is not a good idea to determine whether a pixel is the edge or not strictly because the spectral features of pixels on the shadow boundaries are not discriminable enough.
In this paper, a soft edge model is raised to present the probability of a pixel being considered as an edge pixel. Thus, pixels with lower probability should be labeled according its spectral features and the neighbor interaction, which tend to obtain more homogeneous area. Comparatively, label assigned to pixels which have higher edge probability should lead to more clear boundaries. It means that we should balance the influence of different pixels in the iterative procedure. So, there came a new object function, which is defined to ensure that different roles are assigned to more likely edge pixels and the other ones. Given all that, there are two main contributions in this paper. One is the soft edge model, and the other is a new object function based on which an iterative shadow detection method is proposed. This paper is organized as follows. In Section 2, we will describe the soft edge model and the proposed clustering method with edge constraints, which followed by the analysis of experiments on remote sensing images in Section 3. Finally, conclusions are drawn in Section 4.

Soft Edges Model
As mentioned in section 1, a natural way to describe shadow edges is using the soft manner. Therefore, an essential indicator must be defined to measure the probability of pixels being considered as shadow edges.
Let Y denote an intensity image defined on a m n × rectangular lattice set, should be considered as edges. Therefore, many researches explored shadow detection method using threshold of gradient. Taking "Canny edge detection" as example, two thresholds of gradient are used to determine whether a pixel is an edge one or not. We can introduce these two thresholds into the soft edges model, shown as: In which, i grad is the gradient of ith pixel, l θ is the larger threshold in Canny detection while s θ is the small one. Therefore, i g indicate the degree of ith pixel belonging to an edge in view of gradient.
In actual images, not all pixels with higher gradient value are true edges, for example, noises. To exclude this, a shadow detection using intensity features will be employed as the initial detection and then neighbor pixels would be exploited.
Generally, neighbors of a noise pixel belong to the same class, which means they should have the same label. In other words, the more shadow neighbors a pixel has, the lower edge probability it should be assigned. Aiming at achieving an edge probability for each pixel, the soft edge indicator should take the two terms into account.
So, the indicator can be defined as: where i ∂ denote the neighborhood of i th pixel while N is the number of the neighbors. α is a factor who can balance the power of two terms.

Shadow Detection Method with Soft Edges
Aiming at partitioning an image into shadow areas and non-shadow areas, the procedure of shadow detection can be treated as a process of image labeling. As known, HRMFs can be found in most image labeling methods for the excellent capability of spatial description. At the same time, Fuzzy C-means (FCM) clustering is also one of the most widely used algorithms which can retain enough information from the original images compared to threshold methods. Hereby, HMRF-FCM [13] combining the benefits of HMRF and FCM to deal with the fuzziness and region homogeneity of the labeled images becomes a natural choice.
HMRF-FCM incorporates the HMRF into FCM by understanding HMRF in a fuzzy way. It treats kth HMRF models as a fuzzy classification. The fuzzy classi- our shadow detection task). And an iterative procedure is carried out to updating the membership matrix. Our proposed soft edge model is employed in the iterative updating procedure to impose edge constraints. As mentioned above, labels of edge pixels should lead to more accurate boundaries while that of shadow ones should be of benefit to region homogeneity.
Considering that boundaries generally are continuous along a certain direction, a novel membership is defined as: In which, s i ∂ denotes neighborhoods of ith pixel along direction s. ( ) δ i is a function denoted as: (3), along direction s, the more same labeled neighbors the pixel has, the larger value be assigned to ki b . In other words, it is also a membership of i th pixel to the k th cluster. But the main consideration about ki b is that the label k given to i th pixel should make the pixel have more same labeled neighbors along one direction. The direction s is determined by: ( ) Based on this, the objective function of this iterative clustering procedure can be defined as: ( )  (5) in which, And ki π is the pointwise prior probabilities of the HMRF model states, which can be denoted as: To sum up, the proposed algorithm for shadow detection comprises the following steps.

Data
To evaluate the performance of proposed shadow detection method, we chose three pieces of remote sensing urban images among which the main variation is the complexity of scenes contents (Figures 1-4).

Compression Approaches
To verify the superiority of the proposed shadow detection method to the ones without edge constraints, three methods are employed as the competitors: bithreshold method [7], PCAHSI [8], and soft Shadow Detection method [12].

Bithreshold method
Bithreshold method tries to use two spectral features of shadow areas. Firstly, transform the image into HIS color space, compute the normalized difference of intensity ( ) and saturation ( ) components, and obtain the initial detection by its threshold. Then, get the detection result of channel by histogram threshold.
The final result is obtained by performing AND operation on two detected results mentioned above.

Experiments and Discussion
In this section, the performance of the proposed method is verified by experimental results. Some parameters in our experiments were chosen empirically. In

Visual Comparison
Experimental results on the first image using different methods (bi-threshold, PCA method, soft shadow method, the proposed method) are shown in Figure   5. Figure 5(e) is the ground truth of image 1.
Obviously, there are many misdetection in Figure 5(a) and Figure 5

Quantitative Comparisons
To obtain a quantitative comparison between different algorithms, both recall and precision are employed as the performance metrics. Recall represents how many true shadow pixels have been detected as shadow pixels, which is denoted by where t N is the number of true shadow pixels, while c N is the number of pixels detected as shadows correctly. c N is computed by performing AND operation on the detected result and the true shadow mask.
Precision indicates that in the detected shadow pixels, how many correct ones are there. It can be denoted as: where d N is the number of pixels labeled as shadow. From the definition, it is easy to conclude that the recall favors over detection and the precision favors under detection. That is to say, high recall combined with a low precision means over detection shadows. We measured the recall and precision of each method and listed their values in Table 1. From the quantitative results as shown in Table 1, it is easy to conclude that the proposed method can obtain more accurate shadow detection results than the competitors.

Conclusion
In this paper, in order to add edge constraints into shadow detection, a soft edge

Conflicts of Interest
The author declares no conflicts of interest regarding the publication of this paper.