A new technique using fuzzy in a recursive fashion is presented to deal with the Gaussian noise. In this technique, the keyframes and between frames are identified initially and the keyframe is denoised efficiently. This frame is compared with the between frames to remove noise. To do so the frames are partitioned into blocks; the motion vector is calculated; also the difference is measured using the dissimilarity function. If the blocks have no motion vectors in the block, the block of value is copied to the between frames otherwise the difference between the blocks is calculated and this value is filtered with temporal filtering. The blocks are processed in overlapping manner to avoid the blocking effect and also to reduce the additional edges created while processing. The simulation results show that the peak signal to noise ratio of the new technique is improved up to 1 dB and also the execution time is greatly reduced.
Videos and photographs play vital roles in everyday life. They are the non-destroyable and evergreen gift to remember the memories. The applications of videos and photographs are uncountable i.e., they are used to control the traffic via surveillance cameras, to suggest about a disease via video conferencing, to think about alternate to earth via satellite and also to secure our homes, offices, and streets. This may be the strong evidence to more crimes too. For example, consider that an accident takes place. A stranger who comes back can take a photograph or video of that incident. That photo contains the registration number of the vehicle, labels, the wardrobe, or sometimes the model of the vehicle etc. Unfortunately the number and label are shown but not clear. The reason is that, the stranger might have used a lower end mobile phone, or the environment will be polluted, etc. They are unwanted pixels namely noise will be added in the photograph. Hence an intelligent technique is needed to remove the unnecessary details in the photo or video moreover to get the clear and zoomed view of the number or label without spending a lot of precious time. Moreover very uncommon photos taken in some situations will get corrupted. To restore the photos, this application is very helpful.
To develop such an application, a tool with the power of converting the human thinking into mathematical model is needed. Such a tool was introduced by Lotfi Zadeh with the name of fuzzy sets in the year 1965. Generally, human thinking will be based on approximation. Such approximation can be easily represented using this fuzzy. Because whatever option we are thinking is exactly converted to rules by the developer. Over the last several years, an enormous amount of filters also filters using fuzzy tool were developed. According to the change of everyday lives, there is a gap which needs further development in all the applications. Because change is the only word that does not get changed anytime.
While looking into the previous developments, the developers really proved themselves. First, they had a close look into the pixels and processed them in a 3 × 3 matrix, or 5 × 5 matrix i.e. in a window manner. By modifying the intensity of the pixels based on averaging the neighbouring pixel values or the median value of the pixels or the center pixel values in the window, they removed noise from the photographs. They named the technique as spatial filtering [
While taking into account the videos, they are sequence of frames. The frame rate is different for different displays. It may be 20 fps or 30 fps according to the application i.e., 20 frames or 30 frames are displayed per second. If nose is present in the video, the spatial filter should be applied to each and every frame. It will be a time-consuming process and also it produces some delay. So the developers like Yan and Yanfeng, Rakhshanfar and Amer had a close look into the details about temporal [
Both techniques contain their positive and negative sides. By integrating these two techniques namely spatio- temporal filtering [
Some authors changed the entire processing domain and initiated the concept of wavelets [
Again the researchers moved on to the new world of fuzzy [
In the present work a new technique based on the above said fuzzy logic, is initialized. Here the added advantage is obtained in the output and also the execution time is reduced much better. To check the pros and cons of the algorithm, three algorithms are selected.
The first algorithm which is an extension of fuzzy directional impulsive noise removal filter is Fuzzy Directional Adaptive Recursive Temporal Filter for Gaussian denoising of color video sequences (FDARTF_G). It is a kind of spatio temporal filter. A different approach takes place here. That is, the first frame in the video is denoised using the existing mean weighted filter. This frame is taken as the reference frame. In the subsequent frames, the difference in the movement of images between the reference frame and the next frame is calculated using angle deviation. The fuzzy rules are fired to maintain the minute details of the images that are moving. Recursively processing the subsequent t + 1 frames leads to good result in time saving.
The next hybrid algorithm to be compared with the new algorithm is Color video Block Matching 3D (CBM3D) by Maggioni et al. This algorithm integrates the spatial and transform domain for processing. Using three steps, the denoising of video is done. In the first step, the frames are divided into blocks. The blocks contain same within the frame and in the consecutive frames are selected, tracked via motion trajectory and placed in a volume. These blocks are denoised via transform domain and the reverse is obtained to get the original output. To store and process the blocks more memory is required. If there are very large numbers of same blocks, it will slow down the execution.
The recent algorithm Fuzzy Multi channel Additive Noise Suppression (FMANS_2) algorithm has been developed by Ponomaryov V. et al. This algorithm processes the video in three stages. During the initial stage, the noise pixels are identified and refined. In the second stage, two adjacent frames are processed in a manner by calculating the differences between these two frames. In the final stage, the precise details are processed individually.
The proposed model is given in
important feature of video i.e., keyframe concept. The keyframe contains the meaningful data about a particular shot of the video. Hence applying the efficient denoising algorithm on the keyframes, leads to easier denoising of subsequent frames. The technique is that the first between frame and the first denoised keyframes are taken. The motion vectors are calculated using one of the best block matching algorithm. If the motion estimation is zero, the block difference is analyzed. If it is also null, then the block is copied to the between frame else the displacement is added with the block value and then pasted. If it is equal to one, it is to be analyzed that whether the value is due to edge pixels or not. This is identified by writing a fuzzy rule using angle difference between the center pixel and the neighbouring pixel in the between frame. If it is concluded that the change is because of edge pixel, edge preservation is applied. Otherwise the displacement is again added. Thus the image details are maintained. This loop is continued until the next frame is recognized. To reduce the box effect, the blocks are overlapped.
The proposed filter passes through three steps (i) Keyframe extraction (ii) Keyframe restoration and (iii) Between frame restoration to perform the filtering in a fast way.
To extract the keyframes, abrupt shot change detection technique is enough because the false detection won’t affect the processing of denoising. The necessity of using the keyframes is that they contain more detail about the particular scene. Thus while compressing the video they are in need of large memory to store and retrieve. Nowadays there are more video coding standards available to compress the video and to retrieve the video into their original size. They give the details of number of blocks in the video, motion estimation vectors, etc., to help while playing the compressed video. These details are very useful and external motion estimation and compensation is not needed. Thus video denoising concept integrated with video coding standards reduces the bottleneck of execution time. If there is no shot change in the given video, according to the frame rate, the key frames can be set by the user. Otherwise to identify the scene change, the difference between successive frames within a block is used as a simple measure and this value is used to detect the keyframes from the given video. Initially the frames are divided into non overlapping number of blocks for the use of fast computation. The dissimilarity measure, the mean absolute value differences is calculated because of reduced computational complexity within a block is calculated. Mean Absolute Frame Differences (MAFD) between the key frame and the subsequent frames plays dual role. One is that it is used for calculating the displacement (d) and the next one is that for key frame detection. Each block
where
The keyframe is processed with the existing efficient denoising algorithm. The function
If ((min(angle deviation between the center and diagonal pixels) > 0.1) OR (min(angle deviation between the center and adjacent pixels) > 0.1) then noise present in the block else the edge details present in the block.
By using the above rule executing in a 3 × 3 window the exact block is selected and processed accordingly.
In the between frame, decision to work on noisy pixel or edge pixel restoration is taken using the above said rule and processed accordingly. If it is noisy, then any existing temporal restoration algorithm is applied else the edge feature of the image is preserved by adjusting the pixel values related to the neighbouring pixel value. This kind of processing uses 3 × 3 matrix to avoid the blocking structure and the creation of additional edges generating ghosting effect. This process is repeated while encountering a new keyframe in the video sequences.
Input: RGB noisy color video Z
1. Extract the keyframes from the given video
If
The video frames in order
2. Denoise the keyframe using existing denoising method
3. Calculate the motion vectors and block differences
a. For mv = 0 and no difference:
Copy and paste the values from the keyframe in the same location
b. For mv = 0 but difference in blocks
The difference is added with the keyframe values and then pasted in the same location
c. For mv > 0 angle difference is calculated to decide for noise pixel or edge pixel
4. Check for the next keyframe. If yes, go to step 2, else repeat step 3
Output: The denoised video sequences in the same order.
This section presents the performance obtained by the proposed algorithm and it is compared with the existing algorithms (CBM3D, FDARTF_G and FMANS_2) which are presented earlier. The video sequences are corrupted by various levels of Gaussian noise VAR = 0.001, 0.01, 0.02, 0.03 to corrupt the input sequences. The original sequences of Flowers, Miss America, Chair and also the Gaussian noise corrupted sequences are given in
video sequence for the existing methods and for the proposed method. It demonstrates that the proposed technique performs better for all the levels of Gaussian variance.
The denoising work is differently projected in this paper. Processing the keyframes and between frames recur-
Filters Noise (VAR) | Proposed Algorithm | CBM3D | FDARTF_G | FMANS_2 | ||||
---|---|---|---|---|---|---|---|---|
PSNR | MAE | PSNR | MAE | PSNR | MAE | PSNR | MAE | |
0.001 | 36.81 | 4.52 | 36.46 | 4.74 | 35.68 | 5.36 | 36.83 | 4.62 |
0.01 | 33.97 | 8.43 | 32.53 | 8.79 | 31.69 | 9.7 | 32.95 | 8.65 |
0.02 | 30.51 | 10.43 | 29.65 | 10.42 | 28.84 | 10.31 | 30.07 | 10.48 |
0.03 | 27.9 | 10.91 | 27.39 | 12.03 | 26.26 | 12.82 | 27.83 | 11.83 |
Filters Noise (VAR) | Proposed Algorithm | CBM3D | FDARTF_G | FMANS_2 | ||||
---|---|---|---|---|---|---|---|---|
PSNR | MAE | PSNR | MAE | PSNR | MAE | PSNR | MAE | |
0.001 | 37.91 | 4.52 | 36.44 | 4.72 | 35.66 | 5.34 | 36.80 | 4.60 |
0.01 | 33.57 | 8.61 | 32.51 | 8.76 | 31.67 | 9.60 | 32.93 | 8.63 |
0.02 | 30.51 | 10.39 | 29.63 | 10.40 | 28.82 | 11.30 | 30.05 | 10.46 |
0.03 | 27.89 | 11.79 | 27.37 | 12.01 | 26.24 | 12.80 | 27.81 | 11.81 |
sively throughout the video gives a pleasant vision without any delay. At any cost, only one frame is processed considering all the image features and often, the variations between blocks of two frames are calculated. If there is no motion vector and also no variations between the blocks, the values are just copied and pasted to the between frames. To effectively denoise the forthcoming frames with reference to the keyframe, this process is executed recursively till the end until a new keyframe is reached. This technique is very much useful for real-time processing and this technique can be adapted for getting the super resolution images also. Instead of executing the denoising algorithm in the key frames, any existing super resolution algorithm can be applied to get the high resolution frames. If this denoising technique is combined with the video coding standards, there will be a great reduction in the execution time. The denoising of keyframes is considered as the main part in this technique and it is very useful for the video summarization applications of various surveillance videos, sensor networks etc. This technique can also be very useful for video indexing on internet.
V. Sasikala,N. UmaMaheswari,R. Venkatesh, (2016) Fuzzy Logic Recursive Gaussian Denoising of Color Video Sequences via Shot Change Detection. Circuits and Systems,07,1560-1568. doi: 10.4236/cs.2016.78136