Comparing Pan-Sharpening Algorithms to Access an Agriculture Area: A Mississippi Case Study ()
1. Introduction
Satellite systems have become the pillar for acquiring imagery of the Earth’s surface. High, medium, and coarse spatial resolution satellite imaging systems exist, and they all play a role in monitoring the planet. Modern-day satellite systems often carry payloads that collect data at different spatial and spectral resolutions. The high spatial resolution panchromatic imagery obtained by these systems can be merged with the lower spatial resolution multispectral images to produce a new product (pan-sharpened) containing the spatial and the spectral resolutions of the panchromatic and multispectral images, respectively [1] [2] , resulting in new imagery with more detail than the original multispectral or panchromatic images [1] [2] [3] .
Currently, pan-sharpening algorithms are available in commercial and open-source software. The main goal of image fusion is to merge the imagery without affecting its data quality [1] [4] . Improper fusion of panchromatic and multispectral imagery often results in color distortions and blurriness of the newly merged imagery [5] [6] . Several literature reviews have been published on image fusion [7] [8] and on which algorithms may perform the best in certain circumstances [3] [9] . Pan-sharpened satellite imagery has been used in environmental sciences [10] [11] , urban planning [12] , agriculture [13] [14] , and military and surveillance [15] applications. Its success and failures often hinge on the algorithm’s ability to handle complex landscapes.
Fusion methods have been divided into component substitution (CS) and multiresolution analysis (MRA) based techniques. The component substitution methods transform the spectral content of the multispectral imagery and then substitute the panchromatic band or some other component for one of the transformed components. Then the imagery is back-transformed to the normal image space. Common image fusion approaches classified as CS techniques are the intensity-hue-saturation [16] [17] [18] , principal component analysis [14] , and Gram-Schmidt [19] [20] [21] spectral sharpening methods. Adaptive component substitution methods have also been developed for fusing imagery [22] [23] .
The MRA procedures inject the spatial detail derived from the decomposition of the panchromatic image into the up-sampled multispectral image. À Trous wavelet transform [24] , undecimated or decimated wavelet transform [25] , the Laplacian pyramid [18] , contourlet transform [10] [11] , high-pass filtration, and curvelet [26] are common MRA approaches. MRA fusion techniques often maintain the structural integrity of the image at the cost of spatial distortions. Also, hybrid methods integrate component substitution and multiresolution analysis [27] [28] . New procedures such as hyperspherical color sharpening [29] and improvements to the contourlet transform [30] have been established to improve the pan-sharpening of satellite imagery.
Agricultural regions in rural areas can consist of landscapes dominated by vegetation to mixtures of farmland and buildings. High, medium, and coarse-resolution satellite imagery, even pan-sharpened imagery, have been used to study rural landscapes [31] . Nevertheless, more detail is needed on the performance of pan-sharpened algorithms because satellite imagery is often the primary image source available to people in rural areas. Furthermore, information is lacking in Mississippi on the value this imagery has for assessing landscapes in rural areas. This study aimed to determine the effect of landscape variability in agricultural areas on high-resolution pan-sharpened imagery. The study compared commonly available pan-sharpened algorithms used to fuse high spatial resolution satellite imagery with coarse spatial resolution multispectral imagery. The area of interest contained agricultural plots, woody areas, roadways, and buildings commonly found in rural areas of Mississippi.
2. Materials and Methods
Worldview 3 (Maxar technologies) satellite imagery was acquired of an agricultural area near Stoneville, Mississippi, USA, on June 14, 2022 (Figure 1). The image acquisition was part of an ongoing research project conducted by the United States Department of Agriculture, Agricultural Research Service Scientists to monitor crop growth and productivity in the region. The vendor, atmospherically and radiometrically, corrected the imagery. The satellite nine-band image bundle characteristics were as follows: panchromatic—0.3 m spatial
![]()
Figure 1. Map of the United States, Mississippi, and the closest city to the study site. Inset—image of the study area.
resolution, multispectral—1.2 m spatial resolution, radiometric resolution—11-bits, spectral resolution (nm)—450 - 800 (visible to near-infrared, panchromatic band), 400 - 450 (coastal), 450 - 510 (blue), 510 - 580 (green), 585 - 625 (yellow), 630 - 690 (red), 705 - 745 (red edge), 770 - 895 (near-infrared 1), and 860 - 1040 (near-infrared 2).
A subset of the panchromatic (832 rows by 832 columns) and multispectral imagery (3328 rows by 3328 columns) was extracted for further study. The study area contained soybean (Glycine max L), sorghum (Sorghum bicolor L.), and corn (Zea mays. L) plots, natural vegetation, a few houses, and roads. The image represented a common but complex scene often encountered in rural areas of Mississippi, USA.
The imagery of the area of interest was processed in two separate experiments [5] [7] [26] , full resolution experiment (Experiment 1) and reduced resolution experiment (Experiment 2). Experiment 1 involved pan-sharpening the 1.2 m multispectral resolution imagery to 0.3 m spatial resolution. Then the pan-sharpened imagery was down sampled to 1.2 m. For quantitative analysis, the down-sampled multispectral image quality was compared to the original multispectral imagery. For Experiment 2, the panchromatic and multispectral subset was subjected to a low-pass filter. Then the panchromatic and multispectral image subsets were resampled to 1.2 m and 4.8 m spatial resolution, respectively. The resampled multispectral imagery was pan-sharpened to 1.2 m spatial resolution. Finally, its image quality was compared to the original multispectral imagery with a spatial resolution of 1.2 m. Resampling of the imagery was completed with QGIS software (3.22.8-Białowieża [32] ).
The imagery was fused with the PanFusion software (version 2.4). The software and its instructions are freely available to the public (https://www.pansharp.com/applications/). It contained 18 pan-sharpening algorithms for fusing multispectral and panchromatic imagery. For this study, 17 fusion methods were evaluated: band-dependent spatial detail (BDSD [2] [3] ), à trous wavelet transform (ATWT [33] ), additive wavelet intensity method (AWLP [3] [34] ), smoothing filter-based intensity modulation (SFIM [35] ), generalized laplacian pyramids with modulation transfer function context-based decision injection scheme (MTF_GLP_CBD [36] [37] ), gaussian modulation transfer function match filtered with high-pass modulation injection model (MTF_GLP_HPM [9] ), modulation transfer function generalized laplacian pyramid (MTF_GLP [38] ), high pass filter (HPF [39] ), Gram-Schmidt (GS [3] [40] ), hyperspherical color space (HCS [41] ), à trous wavelet transform with the injection model 3 (ATWT_M3 [42] ), à trous wavelet transform with the injection model 2 (ATWT_M2 [42] ), local mean matching (LMM [43] [44] ), Brovey [45] , intensity hue saturation (IHS [46] ), optimize high-pass filter (HPFC [47] ), and local mean and variance matching (LMVM [43] [44] ). The GS adaptive algorithm was not evaluated because of a screen error message that could not be corrected. The following were the input parameters used in PanFusion to create the pan-sharpened imagery: panchromatic image, the multispectral images (8 images), interpolation of multispectral images-nearest neighbor, resolution ratio—4 (i.e., pixel resolution ratio between panchromatic and multispectral images), sensor type-generic, and output datatype-auto.
Three quality indices were calculated with the ImAnalysis (version 1.55) software, a companion software to the PanFusion software, to test the quality of the fusion methods. It measures the following eight quality indexes: Bias, CC (Correlation Coefficient), DIV (Difference in Variance), Entropy Difference, ERGAS (Erreur Relative Globale Addimensionelle de Synthese), UIQI (Universal Image Quality Index), RASE (Relative Average Spectral Error), and RMSE (Root Mean Squared Error). Researchers have commonly used ERGAS, Bias, and UIQI indices to assess the image quality of pan-sharpened imagery [23] [48] [49] ; thus, they were the three metrics accessed in this study.
ERGAS (French acronym, relative dimensionless global error in synthesis) is a normalized global spectral index that indicates the distortion of the pan-sharpened image compared to the reference multiband image. The smaller the value, the better [48] . Low values represent a high similarity between the fused multispectral and original imagery.
Bias measures the mean difference between the reference and pan-sharpened images [49] [50] . Values close to zero represent minor differences between the former and the latter.
The UIQI index [51] is derived from a function of the difference between the mean of the panchromatic and multispectral images, the standard deviation of the differences, and the spatial resolution of the input and output of the panchromatic and multispectral imagery. The UIQI incorporates an estimate of the correlation coefficient and the difference in the luminance contrast between images. Its value ranges from −1 to 1, with 1 representing the best fidelity to the reference [4] [48] .
According to the instructions for the ImAnalysis software, the user must select a search neighborhood for the image processing and the ratio between the multispectral and panchromatic image pixels. For the search window, the software instructions recommended a value between seven and nine and that an odd number worked best. Thus, a search window of seven was selected for this study. The ratio was 1 to 4, representing the panchromatic image pixel size compared to the multispectral image size. After processing the data, the ImAnalysis software stored the results in a Microsoft Excel, xls. file.
The original multispectral imagery was compared with all images derived with the fusion algorithms for qualitative imagery assessment. For display purposes in the results section, the traditional color composite and the false color, color infrared composite were used for comparison. Those two-color composites are commonly viewed for the qualitative assessment of vegetation areas. The traditional color composite was created with the red, green, and blue images; the color infrared composite was derived from the near-infrared 1, red, and green images. The composite imagery was created with the QGIS software.
3. Results
3.1. Qualitative Results
A qualitative assessment of the conventional color and color infrared composite images indicated that the appearances of the pan-sharpened imagery derived from experiments one and two were similar; therefore, the product produced from experiment one was used as the representative images of both investigations. Furthermore, similarities were observed between the fusion processes, thus, a representative image similar to more than one fusion method was presented for further discussion.
Figures 2-5 show the panchromatic, the original traditional color and color infrared composites, and the pansharpened multispectral images. The pan-sharpening algorithms improved the spatial image resolution (Figures 2-5). The spatial enhancement was not the same for all algorithms tested. The most extreme case of spatial enhancement was observed for the HPFC method, whereas the ATWT_M3 fusion technique represented the least spatial enhancement (Figures 2-5). Other fusion methods that had similar spatial results to HPFC were ATWT, GS, HPF, IHS, MTF_GLP_CBD, MTF_GLP_HPM, and MTF_GLP. Similar spatial detail was observed between ATWT_M3 and LMVM, and ATWT_M2, LMM, HCS, Brovey, BDSD, SFIM, and AWLP.
![]()
Figure 2. (A) Panchromatic image (0.3 m, spatial resolution), (B) traditional color raw imagery (1.2 m, spatial resolution), and traditional color fused image (0.3 m, spatial resolution); (C) à trous wavelet transform with the injection model 3; (D) local mean and variance matching. The images include experimental plots, natural vegetation, bare soil, water, roadways, and buildings. Imagery acquired by Maxar Technologies.
![]()
Figure 3. Traditional color fused image (0.3 m—spatial resolution): (A) optimize high-pass filter; (B) intensity hue saturation; (C) Gram-Schmidt; (D) smoothing filter-based intensity modulation. The image includes experimental plots, natural vegetation, bare soil, water, roadways, and buildings. Imagery acquired by Maxar Technologies.
![]()
Figure 4. (A) Color infrared raw imagery (1.2 m, spatial resolution) and color infrared fused image (0.3 m, spatial resolution); (B) à trous wavelet transform with the injection model 3; (C) local mean and variance matching; (D) optimize high-pass filter. The image includes experimental plots, natural vegetation, bare soil, water, roadways, and buildings. Imagery acquired by Maxar Technologies.
![]()
Figure 5. Color infrared fused image (0.3 m, spatial resolution): (A) intensity hue saturation; (B) Gram-Schmidt; (C) smoothing filter-based intensity modulation. The image includes experimental plots, natural vegetation, bare soil, water, roadways, and buildings. Imagery acquired by Maxar Technologies.
The fusion process resulted in a spectral distortion in some images, such as IHS, Gram-Schmidt, and HPFC (Figures 2-5). The spectral distortion errors included changing the color of vegetation and the other land cover features. The only other fusion method that resulted in changes in the image color was Brovey (not shown). Spectral distortions were not evident in the ATWT_M3 and LMVM (Figure 2 and Figure 4) pan-sharpened imagery. ATWT, AWLP, BDSD, HCS, HPF, LMM, MTF_GLP_CBD, and MTF_GLP fused images (not shown) appeared similarly to ATWT_M3 and LMVM.
The SFIM fusion method added artifacts to the imagery (Figure 3 and Figure 5). The artifacts were more apparent in areas containing shadows and transition zones between one cover type to another. They were more noticeable on the traditional color composite image than on the color infrared composite image, indicating the images (red, green, and blue) used to develop the traditional color composite were more affected by the SFIM fusion process. MTF_GLP_HPM fused imagery (not shown) contained artifacts like those observed on the SFIM pan-sharpened product.
3.2. Quantitative Results
Table 1 summarizes the quality index results for Experiment 1. For all fusion methods, the bias values were relatively low and close to zero, indicating the
![]()
Table 1. Quality index results of pan-sharpened experiment (experiment 1) of the agricultural landscape imagery. The multispectral image (1.2 m spatial resolution) was pan-sharpened (0.3 m spatial resolution) and then resampled to the resolution of the original multispectral image (1.2 m spatial resolution). The index values are an average of the eight spectral bands of the Worldview 3 satellite imagery of the study area.
ATWT_M2—à trous wavelet transform with the injection model 2, ATWT_M3—à trous wavelet transform with the injection model 3, ATWT—à trous wavelet transform, AWLP: additive wavelet intensity ratio, BDSD: band dependent spatial detail, GS: Gram-Schmidt, HCS: hyperspherical color space, HPF: high pass filter, HPFC: optimize high-pass filter, IHS: intensity hue saturation, LMM: local mean matching, LMVM: local mean and variance matching, MTF_GLP_CBD: generalized laplacian pyramids with modulation transfer function context-based decision injection scheme, MTF_GLP_HPM, gaussian modulation transfer function match filtered with high-pass modulation injection model, MTF_GLP: modulation transfer function_generalized laplacian pyramid, SFIM: smoothing filter-based intensity modulation. Quality indices: ERGAS: Erreur Relative Globale Addimensionelle de Syntheses, UIQI: universal image quality index. *: best score per index, **: 2nd best score per index, ***: worst ranking per index.
histogram was not shifted drastically between the original multispectral imagery and the pan-sharpened multispectral imagery. The top-ranked fusion algorithms for bias were ATWT_M2, ATWT_M3, ATWT, AWLP, and HPF. The second-best-rated algorithms were HCS, LMM, LMVM, and MTF_GLP_CBD. The worst-ranked fusion algorithm for bias was the HPFC fusion method. A word of caution for these results. If the values had more than three zeroes after the decimal, the value was rounded to zero, resulting in the ties observed in the quality index tables.
The highest-ranked fusion method, according to ERGAS was HCS, and ATWT_M3 was a close second. HPFC was ranked the worst, followed by IHS, with ERGAS values of 8.73 and 7.71, respectively. The top, second, and worst-ranked fusion methods for UIQI were ATWT_M3, HCS, and HPFC, respectively.
Experiment 2 results are shown in Table 2. The bias values were also low, with the top-ranked fusion algorithms being HCS and LMM. ATWT_M2, ATWT_M3,
![]()
Table 2. Quality index results of the pan-sharpened experiment (Experiment 2) of the agricultural landscape imagery. The multispectral and panchromatic imagery were degraded; the multispectral image was pan-sharpened and compared to the original. The index values are an average of the eight spectral bands of the Worldview 3 satellite imagery of the study area.
ATWT_M2: à trous wavelet transform with the injection model 2, ATWT_M3: à trous wavelet transform with the injection model 3, ATWT: à trous wavelet transform, AWLP: additive wavelet intensity ratio, BDSD: band dependent spatial detail, GS: Gram-Schmidt, HCS: hyperspherical color space, HPF: high pass filter, HPFC: optimize high-pass filter, HIS: intensity hue saturation, LMM: local mean matching, and LMVM: local mean and variance matching, MTF_GLP_CBD: generalized laplacian pyramids with modulation transfer function context-based decision injection scheme, MTF_GLP_HPM: gaussian modulation transfer function match filtered with high-pass modulation injection model, MTF-GLP: modulation transfer function-generalized laplacian pyramid, SFIM: smoothing filter-based intensity modulation. ERGAS: Erreur Relative Globale Addimensionelle de Syntheses, UIQI: universal image quality index. *: best score per index, **: 2nd best score per index, ***: worst ranking per index.
LMVM, and MTF_GLP were ranked second. HPFC performed the worst. LMM, LMVM, and MTF_GLP_HPM achieved the best, second-best, and worst ERGAS scores, respectively. Other fusion algorithms posting values close to eight were HPFC, IHS, and SFIM. Four fusion methods were tied for the best UIQI score, ATWT_M3, Brovey, HCS, and LMM. ATWT_M2 and GS were ranked second, and the HPFC fusion method was ranked last.
4. Discussion
Worldview 3 pan-sharpened satellite imagery was evaluated as a tool for assessing a common but complex agricultural landscape found in rural areas of Mississippi. The image fusion resulted in new imagery with more detail than the original multispectral or panchromatic images. Overall, ATWT_M3 and HCS were often ranked as one of the best fusion algorithms for this study (Table 1 and Table 2). It was easily seen on the synthesized traditional color and color infrared composite imagery of ATWT_M3 (Figure 2 and Figure 4) that its image integrity was almost identical to the original imagery. Its colors did not appear washed out like the IHS (Figure 3 and Figure 5), GS (Figure 3 and Figure 5), and Brovey (not shown) fusion methods and, to a lesser extent HPFC method (Figure 3 and Figure 4). The SFIM (Figure 3 and Figure 5) and the MTF_GLP_HPM pan-sharpened images were ruled out for further visual assessment because of the additional artifacts added to the images.
Reference [5] compared eleven different pan-sharpening approaches on hyperspectral data and reported the Bayesian sparse method provided the best performance. The results were consistent across three landscapes, a mixed urban/rural scene, a rural area with different crop types, and a rural area with small villages. Furthermore, their study included four pan-sharpening algorithms equivalent to the ones used in the current research: SFIM, MTF_GLP, MTF_GLP_HPM, and GS. They used ERGAS as one of the quality indices to evaluate image quality. For the rural landscape, the order from best to worst was MTF_GLP_HPM, MTF_GLP, SFIM, and GS. Their findings were based on the reduced resolution dataset analysis, equivalent to experiment 2 in this study in which the rankings were MTF_GLP, GS, SFIM, and MTF_GLP_HPM.
Reference [6] in their studies of twenty-one pan-sharpening techniques demonstrated for visible and shortwave infrared Sentinel-2 satellite imagery of diverse landcover types, the MTF_GLP_CBD was ranked best. For their research, sixteen pan-sharpened algorithms were equivalent to the ones evaluated in the current study. Generally, the SFIM, HPF, MTF_GLP_HPM, MTF_GLP_CBD, and MTF_GLP were ranked in the top ten for the reduced resolution and the full-resolution pan-sharpened datasets. In contrast, this study’s consistent top-ranked pan-sharpened algorithms were ATWT_M3, HCS, ATWT_M2, LMVM, and LMM. Reference [6] also reported artifacts in pan-sharpened imagery created by the SFIM method.
Reference [48] observed in their evaluation of pan-sharpened Worldview2 satellite imagery that different algorithms were ranked best when applied to the degraded and to the original data sets. The findings of this study concurred with those results. Overall, the bias values appeared to be less effective by the resolution of the data, followed by ERGAS. UIQI was effective the most; its overall range was reduced from the worst to the best pan-sharpening method when using the original versus the degraded data sets.
Finally, other researchers have also reported spatial and spectral distortions to satellite imagery subjected to pan-sharpening [3] [6] [23] [48] . The differences observed in the present study and the other discussed studies could be attributed to the differences in the landscapes and differences in the sensor systems used to acquire the imagery for the pan-sharpening process.
5. Conclusion
Worldview 3 satellite imagery subjected to pan-sharpening algorithms can be a great asset in assessing rural agricultural landscapes in Mississippi. The researcher or user is often perplexed about what sharpening algorithm to use. The beauty of the software used in this study was that the practitioner has 18 tools at their disposal and thus can determine the best tools to solve problems from that suite. Future research initiatives should focus on whether the quality index values are better when subsets of the satellite scenes are analyzed in rural Mississippi compared with the whole satellite scene and on how pan-sharpening images affect the classification accuracy of the algorithms used to derive maps.
Acknowledgements
Funding was received from the United States Department of Agriculture. Statements made in this article were the author’s opinion and did not represent the opinion of the Agricultural Research Service. Maxar Technologies acquired the satellite imagery used in the study.