SVM-based Filter Using Evidence Theory and Neural Network for Image Denosing

This paper presents a novel decision-based fuzzy filter based on support vector machines and Dempster-Shafer evidence theory for effective noise suppression and detail preservation. The proposed filter uses an SVM impulse detector to judge whether an input pixel is noisy. Sources of evidence are extracted, and then the fusion of evidence based on the evidence theory provides a feature vector that is used as the input data of the proposed SVM impulse detector. A fuzzy filtering mechanism, where the weights are constructed using a counter-propagation neural network, is employed. Experimental results shows that the proposed filter has better performance in terms of noise suppression and detail preservation.


Introduction
Images are often contaminated by impulse noise during image acquisition (i.e.sensor noise) or transmission (i.e.channel noise) [1].The median (MED) filter is commonly used for suppressing impulse noise due to its simple implementation and efficiency.However, the filter causes smoothing, which blurs fine image details and affects texture.A number of median-based filters have been developed to suppress noise while preserving details.A simple revision of the MED filter is the center-weighted median (CWM) filter, which gives more emphasis to the center pixel in the filtering window [2].The CWM filter improves detail preservation at the expense of lower noise suppression.To balance noise attenuation and image detail preservation, Arakawa et al. and Lin et al. proposed median-type filters based on fuzzy inference rules [3][4].The two filters use an adaptive scheme to set the weighted median value based on fuzzy rules concerning the states of the input pixel.
The median filter and its variations are applied to all pixels of an image irrespective of their level of corruption.These filters thus distort uncorrupted pixels, causing useful information in the image to be lost.This problem is alleviated by a decision-based filter, where the detection of corrupted pixels is carried out prior to filtering.The switching median filter (SWM-I) is the most straightforward decision-based filter [5].Depending on the threshold value, the noisy pixel value is replaced by the output of the MED filter.Noise-free pixels are left unchanged.However, a constant threshold value may not provide satisfactory performance in all circumstances.Thus, Pankaj and Banshidhar proposed the improved adaptive impulse noise suppression filter based on an artificial neural network [6].Unfortunately, the strategy may work well at a pre-assumed noise density level but poorly at other noise density levels.Recently, Lin and Yu proposed an adaptive two-pass median (ATM) filter and Liu et al. proposed an SVM-EPR filter for image denoising based on support vector machines (SVMs) [7][8].The ATM and SVM-EPR filters use SVMs to identify whether an input pixel is noisy.SVM-based filters can preserve image details while suppressing impulse noise, but they sometimes leave noisy pixels undetected or misdetect uncorrupted pixels as noisy.
To get rid of the above-mentioned drawbacks, this paper proposes a novel decision-based fuzzy filter (DBFF) based on Dempster-Shafer (D-S) evidence theory and SVMs for effective noise suppression and detail preservation.The proposed filter comprises an efficient SVM impulse detector and a fuzzy filter.Evidence fusion and SVMs are incorporated into the framework of the proposed impulse detector.For noise cancellation, a fuzzy filtering mechanism, where the weights are constructed using a counter-propagation neural network (CNN), is adopted.Experimental results confirm the effectiveness of the proposed DBFF filter in terms of suppressing impulse noise and perceived image quality.
The rest of this paper is organized as follows.The de-sign of the proposed DBFF filter is presented in Section II.In Section III, experimental results are provided to demonstrate the performance of the proposed scheme.Finally, the conclusion is given in Section IV. w k is defined in terms of the image coordinates symmetrically surrounding the current pixel ( ).

The Decision-Based Fuzzy Filter
x k The filter window with size 2 1 M n   (n is a nonnegative integer) can be given by: { } { ( ) : 1, 2, , , 1, , }, where the input pixel 1 ( ) ( ) is the central pixel.The proposed DBFF filter consists of an SVM detector and a filtering mechanism, as shown in Figure 1.
In the noise-filtering process, if the input pixel ( ) x k is identified as noisy, then the output of the fuzzy filtering replaces the pixel.Otherwise, the pixel is kept unchanged.The fuzzy filtering is performed based on a CNN weight controller to remove any noisy pixels.

The SVM Impulse Detector
The proposed decision-making approach consists of two steps: evidence extraction and fusion, and training of the SVM impulse detector.To apply D-S evidence theory to the SVM impulse detector, the frame of discernment includes two elements: noisy (N) and noise-free (F).The hypotheses to be considered in the D-S formulation are: ,  singleton hypothesis N, singleton hypothesis F, and compound hypothesis NF (we denote { } N F  as NF ).To extract the sources of evidence, we take into account the local features in the filter window { } w k .The following nine feature extraction variables can be defined to obtain the nine independent sources of evidence, respectively.
Definition 1: where MED represents the median operation. where Notably, 1 ( ) x k and 2 ( ) x k of the filter window { } w k are selected as the pixel values closest to that of ( ) x k [4].
Definition 3: The evidence ( ) i c k associated with the local contrast at location k in the filter window { } w k is defined by: where ( )  x k is the mean gray level in the filter window { } w k with size M. ( ) is the associated feature variable [4].
Definition 4: Notably, 1 ( ) c k and 2 ( ) c c k of the filter window { } w k are selected as the variable values closest to that of ( ) c k [4].
Definition 5: where where is the mean gray level excluding ( ) x k in the filter window { } w k .Definition 7: o is defined as the absolute difference in intensity between ( )  x k and ( ); where ( ), ( ), , ( ) are the elements of the filter window { } w k (excluding ( ) How to extract sources of evidence and properly initialize the mass function induced by each source of evidence are key points in the application of D-S evidence theory.Thus, these nine feature variables can be sources of evidence.The strategy for obtaining the mass function values of , , N F and NF of each source of evidence is as follows [9]: where z denotes one the nine variables ( ), ( ), ( ), d k e k c k ( ), ( ), ( ), ( ), ( ), ( ). f k g k h k n k s k t k The three mass functions o m Finally, the evidence vector is given by: After obtaining the evidence vector, { } E k is used as the input data set for the SVM impulse detector.This evidence information is also used in the filtering stage.
To obtain an optimal separating hyperplane for classifying the pixels into noisy and noise-free classes in the SVM impulse detector, we extract evidence vector { } E k in a training image.That is, the optimal separating hyperplane can be obtained through a training process by using a set of supervised class labels (0 or 1) for the training noisy image [11].After training, the nonlinearly inseparable discrimination function is obtained to separate the training data into the noisy or noise-free class.

1) The fuzzy filtering
The noise filtering of the DBFF filter is shown in Fig- ure 1.We incorporate fuzzy filtering into the DBFF filter.The output value ( )  y k of the fuzzy filter at the proc-essed pixel ( )  x k is obtained as follows: To judge whether impulse noise exists at pixel ( ), x k the membership function ( )  k  should take a continuous value from 0 to 1 according the fuzzy rules.Therefore, how to decide the value of the membership function is the main issue for the fuzzy filter.In this work, a novel neural-based learning approach is proposed to solve this problem.
2) The CNN learning algorithm To design ( ), k  the weight controller based on counter-propagation neural network (CNN) shown in Figure 1 is proposed.The framework of the CNN weight controller architecture is shown in Figure 2. The CNN weight controller is composed of a three-layer neural network.The competitive layer which has Q nodes is a simple net for determining the nearest competitive exemplar.The operation and the learning of the network use the counter-propagation algorithm [10].A synaptic prototype weight j w that has the same dimension as the input data { } E k is associated with each neuron j in the competitive layer, as shown in Figure 2.
Each node in the competitive layer competes (winner takes all) based on the given inputs.The CNN uses the Manhattan distance to calculate the similarity between the input and the weight vector.Manhattan distance j U is the difference between the input data { } E k and the weight of the j -th node in the competitive layer.
When all j U are determined, the competitive layer nodes begin to compete.The node with the smallest j U value wins.Suppose that the q-th node is the winner.The output j i of the j-th node in the competitive layer is: The nodes in the competitive layer compete for the input vector to be classified.The node with the largest similarity is the winner; it sends signal 1 to the output node j i .Each training vector is presented to the input layer.The nodes in the competitive layer compete for the input vector to be classified.The CNN network determines the winning node for the training vector in the competitive layer.Then, weight vector , q w from the winner (q-th node) in the competitive layer to the input layer, is updated using the following learning rule. )), where  denotes the learning rate.Notably, the weight vectors to the loser nodes stay unchanged.
In the second learning stage, only the weight connected to the winner is updated.The weight j b from the winner node in the competitive layer (the j-th node) to the node in the output layer is updated as follows.
where ( ) e t is the difference between the desired output and physical output, and  is the learning rate.There- fore, weight j b serves as the weight ( ) k  of the fuzzy filter.

Experimental Results
The optimal separating hyperplane was obtained using training image 'Couple' corrupted by 20% impulse noise in the training process.The tested images were outside the training set to test the generalization capability.Ta- ble 1 shows the accuracy comparison of the SVM impulse detector (for ATM [7], ASVC [11], and DBFF) for some images corrupted by 20% impulse noise.The proposed DBFF impulse detector has the highest classification accuracy.
The image 'Couple' corrupted by 20% impulse noise was used as the training image in the CNN weight controller.The CNN converged after 25 training epochs.The effectiveness of the proposed DBFF filter was assessed by comparing its results with those of existing filters.Figure 3 shows the restoration result comparison on the  image 'Lake' corrupted by 20% impulse noise among MED, ATM and DBFF.Apparently, the DBFF filter is capable of producing better subjective visual quality restored image by offering more noise suppression and detail preservation.

Conclusion
An impulse detector based on SVMs was proposed to judge whether an input pixel is noisy.Sources of evidence are extracted, and then the fusion of evidence based on D-S evidence theory provides a feature vector that is used as the input data of the proposed SVM impulse detector.A fuzzy filtering mechanism, where the weights are constructed using a counter-propagation neural network, is employed.Experimental results show that the proposed DBFF filter achieves much better performance than those of existing decision-based filters in terms of noise suppression and detail preservation.

Acknowledgment
The author is grateful to the National Science Council of the Taiwan, for their support of this research under grant NSC-101-221-E274-010-.

Figure 1 .
Figure 1.The detailed block diagram of the DBFF filter.
o values are sorted in increasing order into the sequence { }.

5
to obtain the combined mass function .

Figure 2 .
Figure 2. The architecture of CNN weight controller.

Figure 3 .
Figure 3. Subjective visual quality of restored image 'Lake' (a) original image, (b) corrupted image by 20% impulse noise and filtered by (c) ATM filter, and (d) DBFF filter.

TABLE I .
ACCURACY COMPARISON OF THE SVM DETECTOR FOR SOME IMAGES CORRUPTED BY 20% IMPULSE NOISE.