A Study of Pansharpened Images Based on the HSI Transformation Approach

A pan-sharpen technique artificially produces a high-resolution image by image fusion techniques using high-resolution panchromatic and low-resolution multispectral images. Thus, the appearance of the color image can improve. In this paper, the effectiveness of three pan-sharpening methods based on the HSI transform approach is investigated. Three models are the hexcone, double hexcones, and Haydn’s approach. Furthermore, the effect of smoothing the lowresolution multispectral image is also investigated. The smoothing techniques are the Gaussian filter and the bilateral filter. The experimental results show that Haydn’s model is superior to others. The effectiveness of smoothing the lowresolution multispectral image is also shown.


Introduction
Recently, there are many Earth observation satellites, which usually provide both high-resolution panchromatic and low-resolution multispectral images.In remote sensing, it is important to acquire high-resolution images.A pan-sharpen technique artificially produces a high-resolution image by fusing a high-resolution panchromatic image and a low-resolution multispectral image [1]. Figure 1 illustrates the outline of generating a pansharpened image.By this pansharpening technique, the appearance of the color image can improve.The pansharpening method is not limited to only a remote sensing image.If there are both high-resolution panchromatic and lowresolution multispectral images, the pansharpening approach is an adaptive technique for every image.Therefore, the fundamental study of a pansharpening technique is believed to contribute to the development of the image processing techniques.J. Zhang reviews current techniques of multi-source remote sensing data fusion [2].In [2], a considerable amount of effort has been devoted to summarize the data fusion techniques of remote sensing data.Moreover, the evaluation of pansharpening fusion methods is reported [3].However, there is little study of a pan-sharpening method of HSI transformation models [4,5].Therefore, in this paper, the effectiveness of three pan-sharpening methods based on the HSI transformation approach is investigated.Three models are the hexcone, double hexcones, and Haydn's methods [6].From the experimental results, the HSI transformation approach based on the Haydn's model works well.Furthermore, the effectiveness of smoothing the low-resolution multispectral image before the HSI transformation is also investigated.The smoothing methods are the use of the Gaussian [4] and bilateral filters [7].From the experimental results, the pansharpened image is shown to be improved by using the smoothing techniques of Gaussian and bilateral filters.

Pansharpened Images
A color is known to be expressed by various color feature models [4,6].In this paper, we used the HSI family of color model for making pansharpened images.The HSI model is a color appearance system.It consists of hue, saturation, and lightness (or intensity).The model based on intensity, hue, and saturation is considered to be better suited for human interaction.As the advantage of the use of the HSI model, each of hue, saturation, and intensity is considered to be independent.In the studies of the pansharpening method, the HSI transformation approach is usually used.Figure 2 shows a flow of pan-sharpening by the HSI transformation.We describe the pansharpened images based on the HSI transformation approach as follows: Suppose that there exist both the low-resolution RGB color image and the high-resolution panchromatic image.Firstly, each RGB element of the low-resolution color image transforms into the HSI element, hue, saturation, and intensity.Secondly, the gray value at the high-resolution panchromatic image is replaced by the intensity obtained from the HSI transformation at the low-resolution color image.Finally, the pseudo high-resolution RGB color image is generated by inverse transformation of the HSI transformation.In this paper, we examine the pansharpened images generated by three representative HSI transformations.Three models are the hexcone, double hexcones, and Haydn's models [6].Note that the intensities (I) of the hexcone, double hexcones, and Haydn's models are defined by the brightness as I = max{R,G,B}, I = (max{R,G,B}+min{R,G,B})/2, and I = R + G + B, respectively.The details of the HSI transformation and inverse transformation are shown in the image processing handbook [6].
Furthermore, in order to improve the pansharpened images, we consider applying the smoothing of the lowresolution multispectral image.The smoothing methods are the use of the Gaussian and bilateral filters.The smoothing technique is expected to use not only one attention pixel but also the color information of its neighbouring pixels.The Gaussian filter is defined as follows: where g(i,j) and f(i,j) denote the intensity of after and before transformation at (i,j) coordinates, respectively.The parameter 2  σ is the spatial variance.The effect of smoothing is determined by a value of σ.The image gets more blurred as the value of σ is getting larger.On the other hand, if the value of σ is small, the image is considered to get sharpen.
The bilateral filter [7] is considered to smooth the image, while preserving the edge information of the image.In smoothing an image, the bilateral filter takes into account of not only pixel difference but also intensity difference.The bilateral filter [7] is defined as Equation ( 2).Here, the parameters 2

Experiments
In the experiments, both the low-resolution color and the high-resolution panchromatic images are generated from an original image.Then, we use 3 types of a ratio.
where the RGB elements of the original image are R, G, and B. On the other hand, the RGB elements of the pansharpened image denote r, g, and b.The smaller value of the RMSE means that the pan-sharpening method works better.Table 1 is the RMSE values of three types of HSI transformation approach for each of a ratio.From Table 1, the Haydn's model gives a smaller RMSE value at any images and a ratio.The results also show that the double hexcones type is usually superior to the hexcone one.From the experimental result, we recommend to use the Haydn's model.

Figure 3. A ratio of the image resolution between the multispectral and panchromatic images.
Table 1.RMSE values of three types of HSI transformation approach for each of a ratio.From the results of Table 1, we focus on improving the Haydn's model.The effects of the parameter value σ of the Gaussian filter are investigated while changing the values from 0.2 to 2.0.On the other hand, for the bilateral filter, the value of 1 σ varies from 0.2 to 2.0, and that of the 2 σ is from 10 to 400.The size of the Gaus- sian and bilateral filter is 3 x 3. Table 2 is the RMSE values of the Gaussian, bilateral, and without filters using the Haydn's model for each of a ratio.The results of the Gaussian and bilateral filters show the minimum RMSE values.Then, Table 3 shows the optimal parameter value which gives the minimum RMSE value.From Table 2, the results of the Gaussian and bilateral filters outperform that of without filter at any images and a ratio.Furthermore, the RMSE value of the bilateral filter is equal to or slightly superior to that of the Gaussian filter.By using the Gaussian or bilateral filters, the pansharpened image can improve.This shows the effectiveness of smoothing the low-resolution multispectral image.From Table 3, the optimal parameter values of σ for Gaussian filter and 1 σ bilateral filter are comparatively very small.This means that the low-resolution multispectral image may be getting sharpen.And, its image quality is considered to improve for making pansharpened images by appropriately adjusting the parameter values.

Conclusion
In this paper, we have examined the effectiveness of pansharpened images using three types of HSI transformation approach.Experimental results show that the Haydn's model is a promising method in the limited study.And the use of the Gaussian or bilateral filters for the low-resolution multispectral image is shown to be effective.Therefore, the Haydn's model with image smoothing techniques such as Gaussian or bilateral filters for the low-resolution multispectral image should be used.In the future study, the effectiveness of other pansharpening methods [3] will be investigated.

1 σ and 2 2 σ 1 σ or 2 2 1 σ
are the spatial and intensity variance.The larger the value 2 σ is, the more an image gets smoothed.Otherwise, the image gets sharpened.The role of 2 is the same as the 2 σ in the Gaussian filter.The effect of smoothing is determined by values of 1 σ and 2 σ .

Figure 1 .
Figure 1. Outline of generating a pansharpened image.

Figure 2 .
Figure 2. Flow of pan-sharpening by the HSI transformation.

Fig- ure 3
shows a ratio of the image resolution between the multispectral and panchromatic images.It indicates that one pixel of the multispectral image corresponds to two by two pixels of the panchromatic image.This shows a ratio 1:2.In the experiment, we change the ratio: 1:2, 1:3, and 1:4.It is fundamentally important to investigate the influences of the ratio on the pansharpened images.For these images, we obtain the pansharpened images using three types of HSI transformation techniques.The effecttiveness of pansharpened images is examined in terms of the RMSE value.The RMSE value is one of the effective measures of the image quality performance.The RMSE value is defined by