Assessment of Supervised Classifiers for Land Cover Categorization Based on Integration of ALOS PALSAR and Landsat Data

Many supervised classification algorithms have been proposed, however, they are rarely evaluated for specific application. This research examines the performance of machine learning classifiers support vector machine (SVM), neural network (NN), Random Forest (RF) against maximum classifier (MLC) (traditional supervised classifier) in forest resources and land cover categorization, based on combination of Advanced Land Observing Satellite (ALOS) Phased Array type L-band Synthetic Aperture Radar (PALSAR) and Landsat Thematic Mapper (TM) data, in Northern Tanzania. Various data categories based on Landsat TM surface reflectance, ALOS PALSAR backscattering and their derivatives were generated for various classification scenarios. Then a separate and joint processing of Landsat and ALOS PALSAR data were executed using SVM, NN, RF and ML classifiers. The overall classification accuracy (OA), kappa coefficient (KC) and F1 score index values were computed. The result proves the robustness of SVM and RF in classification of forest resource and land cover using mere Landsat data and integration of Landsat and PALSAR (average OA = 92% and F1 = 0.7 to 1). A two sample t-statistics was utilized to evaluate the performance of the classifiers using different data categories. SVM and RF indicate there is no significance difference at 5% significance level. SVM and RF show a significant difference when compared to NN and ML. Generally, the study suggests that parametric classifiers indicate better performance compared to parametric classifier.


Introduction
Classification of satellite image is a very significant part of remote sensing image analysis, object and pattern recognition, mapping and monitoring of forest covers and natural resources.The process is commonly utilized for generation of thematic maps like forest, land cover/use maps and spatial pattern maps.Forest and land cover types classification using satellite data has been adopted extensively.Many supervised image classification algorithms have been developed and utilized for forest and land cover mapping, ranging from machine learning algorithms to traditional classifiers [1] [2].Most of the algorithms have been reported to perform reasonably and to enhance the classification accuracy [1] [2] [3].However, it is difficult to identify the best image classification algorithm which suits a particular environment.This is simply because numerous factors tend to affect the results: scheme of classification, satellite data in use, image pre-processing, training and validation sample selection and collection, learning algorithm and post processing approaches and validation techniques [4].For that reason, evaluation of commonly applied machine learning algorithms is essential using same satellite dataset and scheme of classification to aid the selection of suitable algorithm for a particular application.With the advancement of the remote sensing technology, new classification algorithms are developed and invented rapidly.Consequently, it is very important to assess their performance in various kinds of environment using various types of remote sensing datasets [2] The main objective of this study therefore, is to evaluate the capability of the widely applied parametric and non-parametric supervised machine learning algorithms for forest resource and land cover mapping in tropical environment using SAR and optical datasets.Specifically to assess which classification algorithm gives better results using independent and integrated Landsat TM and ALOS POLSAR datasets for categorization of forest resource and land cover mapping.

Study Area
The satellite image utilized for this study is of Bereko and Duru-Haitemba forest reserve in Babati, Tanzania.Lying between latitude 4˚15' and 4˚30' South, and between longitude 35˚35' and 35˚50' East (Figure 1).The area is categorized into six main land cover/use types: water (e.g.lakes), shrubs, natural dense forest and moderate forests [5] (Figure 1).

Dataset and Training Samples
Both Optical and SAR satellite images has been utilized.

Data Processing
ALOS PALSAR HH and HV polarization images were collected in slant range single look complex format.The images were transformed from slant range to ground range resolution using a multi-looking procedure of 9 × 2 (i.e., nine looks in azimuth and two looks in range) [10] [11] [12].Resulting images were of 29.9 m × 27.7 m resolution in range and azimuth consecutively.This procedure improves radiometric resolution and squares the pixels in ground range geometry, which is same as Landsat TM spatial resolution (e.g. 30 m).For speckle reduction a refined Lee spatial filter [13] with a 7 × 7 window size was adapted.The topographic effects on the ALOS PALSAR backscattering was accounted by applying a radiometric terrain correction to convert backscattering in sigma-nought σ˚ value to the improved backscattering in gamma-nought γ˚

Integration of ALOS PALSAR and Landsat TM data
Various input bands were prepared ready for image classification.A multi-sensor integration image fusion approach was adapted [1] [5].The multi-sensor integration approach combines n images in n different layers algorithmically, without creating a new set of images [21] [22] [23]

Classifiers under Study
For classification of forest covers and land cover mapping of the independent bands and integrated ALOS PALSAR/Landsat data and their derivatives Three non-parametric and one-parametric classifiers were tested on their ability.Random Forest [24], Support Vector Machine (SVM) [1] [12] [25] [26] and Neural Network (NN) [27] supervised classifiers.Maximum likelihood (ML) [28] parametric classifier was used for comparison purpose.The ML is a conventional Table 1.Data categorization for various classification set-ups adapted from Deus [5]; SR = Surface Reflectance, AP = ALOS PALSAR, mea = mean, cor = correlation, var = variance, con = contrast, and sec = second moment are GLCM texture derivatives.The SVM is basically a binary class classification method based on machine learning and using support vector in the data classification.[12] [25].Linear, polynomial, radial basis function and sigmoid are the four common kernels available in remote sensing packages.A careful selection of parameter setting can improve the performance of the SVM [29].The Gaussian radial basis kernel function and a penalty parameter of 100 were selected based on trial and error.
However, the kernel and penalty parameter selected are recommended to be the best for land cover classification [29].
NN classifier has arbitrary decision boundary abilities and could adapt to various data types and input structures easily, fuzzy output values and suitable generalization for use when integrating manifold images [27].The classifier benefits from parallel computation, the capability to estimate the non-linear relationship between the input data and desired outputs, and fast generalization ability [1] [30] [31].The NN parameter setting was adapted based on trial and errors.As an activation function a logistic function was chosen, one hidden layer and 1000 training iterations were also designated.
RF is a machine ensemble approach based on classification and regression trees and can be used for both image classification and regression analysis [1] [24] [32] It makes use of multiple self-learning decision trees to parameterize models and use them for estimating categorical or continuous variables [33] [34].The number of trees is a user-defined parameter.RF normally gives higher overall cross-validation accuracies compared to other classification approaches [33] Generally, Non-parametric classifiers yield higher classification accuracy compared to parametric classifiers [1] [30] [31].To run the classification process and assess the potential of parametric and non-parametric classifiers.The data are grouped into three major groups A-C (Table 2), Group A, consists of surface reflectance bands from Landsat 5 TM image and its derivatives (i.e.Vegetation index, GLCM textures and PCA).Group B, comprises of individual ALOS PALSAR backscattering and derivatives.Group C, involve the integration of surface reflectance, backscattering and their derivatives.However, to maximize the overall classification accuracy the best blend of textures, indices and features were identified.The selection of relevant integration bands was carried out based on trial and error classification.

Classification Accuracy Assessment
To test the capability of parametric and non-parametric classifiers a validation dataset was used for accuracy assessment.Three terms that describe the classification accuracy were utilized (i.e.overall accuracy (OA), kappa coefficient (κ)) [35] and F 1 score index [36].The overall classification accuracy is the percentage of the pixels that have been classified correctly in the validation dataset [35].The Kappa coefficient is a metric that compares an observed Accuracy with an expected accuracy.It is used not only to assess a single classifier, but also to assess classifiers amongst themselves.F 1 score index merges producer's and user's accuracy into a fused quantity was computed (Equation ( 1)) [36].Producer's accuracy is used to estimate the omission error to a certain class and it is the probability that a reference site has been classified correctly.User's accuracy is used to estimate the commission error and it is the probability that a pixel classified on the image signifies the actual class on the ground.F 1 score enables a better evaluation of the land cover class-wise accuracies.The score varies between 0 and 1 where by 0 signifies the worst results, and 1 is the best accuracy achieved.
To compare the capability of the four classifiers under study, a two-sample t-test [37] was applied on the overall classification accuracy obtained using different data categories (Table 1).The influence of surface reflectance and backscattering derivatives on the classification accuracy the two-sample t-test was utilized.The two-sample t-test assesses whether two sample means unrelated.A difference in mean indicates that the two samples are dissimilar.The test is normally applied when the test makes use of a small sample size, the variances of two normal distributions are unknown and the experimentation involve a small sample size.

Classification Results Based on the Four Classifiers
The classification results attained based on different data groups (A-C) (Table 1) and tested classifiers are presented in Figure 2 for Overall accuracy, Figure 3 for Kappa coefficients and Table 2 2 and Table 2).For all classifiers when using SAR data water is the only land cover type classified with very higher F 1 score index values followed by shrubs (Table 2).

Evaluation of RF, SVM, NN and MLC Classifiers
The non-parametric classifiers (RF, SVM and NN) are assessed together with the maximum likelihood classifier (MLC) on different data subgroups.Figure 3   backscattering and derivatives, all classifiers displayed poorer performance (Average KC = 0.50), though in most cases machine learning algorithm performed better compared to MLC (Figure 3).In this group, SVM and RF have more or less same performance and there is no substantial difference in their performances at 95% confidence level.Generally, in all data categories, SVM and RF produce better classification value at 95% confidence interval compared to NN.  3).

Discussion
In this research a comparison of supervised learning algorithm using independent and integrated landsat TM and ALOS PALSAR data has been carried out.
The assessment of the performances of the four classifiers under study shows that both parametric and non-parametric classifiers have good performance when using Landsat TM data (Figure 2 and Figure 3).Attarchi and Golaguen [1] attained same results indicating that both parametric and non parametric classifiers performs well for Landsat based surface reflectance and derivatives.
SAR data and derivatives were effectively well classified by RF and SVM classifiers compared to MLC at 95% confidence level.This is probably due to the fact that SAR data and their derived parameters usually do not follow a Gaussian distribution, which is a basic assumption for several classification approaches.Looking on the performance of classifiers based on data category, results in category A, subgroup A1-A3, Landsat surface reflectance and its derivatives indicates that non-parametric classifiers (SVM, RF and NN) as well as MLC performs well (Figure 2 and Figure 3).However NN indicate the lowest performance for data group A2.In category B, subgroup B1-B5 MLC has lowest performance.In this category MLC and NN have the more or less same performance for data group B2 and B4 and there is no significant difference in their performances at 5% confidence level.Subgroup A3 and C1 are the only data categories where all four classifiers indicates nearly similar performances (Figure 2 and Figure 3).In all categories A-C, SVM and RF provides the best performance.In data category C, specifically for C2-C4 NN display lowest performance compared to SVM and RF as well as MLC (Figure 2 and Figure 3).
Attarchi and Gloaguen [1], also indicated a poorest performance of NN compared to SVM and RF.As depicted in Figure 3, SVM and RF appear to have more or less same performance.This is simply because accuracy improvement of land cover mapping by new algorithms are hardly observable [2] [38].Li et al [2] attained more or less similar classification accuracy values for SVM and RF.
The performances of all classifiers within each group are compared at 5% significance level.Comparing all classifiers using the two sample test, the results indicates that there is no statistical significant difference between SVM and RF classifiers at 5% confidence interval.Both SVM and RF classifiers indicate a significance difference when compared to NN and MLC.RF and SVM show a statistically significant different at 5% significance level when compared to MLC.
NN and MLC indicates that there is no statistically significant different at 5% significance level (Table 3).

Conclusion and Recommendations
The potential of parametric and non parametric classifiers has been examined based on integration of Landsat TM and ALOS PALSAR data.All classifiers under study performs well in terms of overall accuracy when using Landsat TM and derivatives, however SVM and RF are superior compared to others.For SAR data SVM, RF and NN performs well compared to MLC.On integration of Landsat and PALSAR data SVM and RF seems to be very powerful compared to were carried out it would be more suitable to select an algorithm for a specific remote sensing application.

Figure 3 .
Figure 3.Comparison of Kappa coefficients for various classification results for different data categories (Table1) and classifiers.The results are based on validation samples.
NN and MLC especially when combining TM derivative, backscattering and GLCM textures.Generally, the overall results indicates the robustness SVM and RF at 5% significance level for land cover classification in tropical area.However, the process of selecting a suitable classifier for a certain task depends much D. Deus DOI: 10.4236/ars.2018.7200457 Advances in Remote Sensing on tradeoffs among classification accuracy, time consumption, and computing resources.Based on the results attained the researcher recommends that, the performance of other classification algorithms, especially object based classification should be tested in tropics and semi-arid environments.This will show their potential ability in terms of differentiating forest resource and land cover mapping.Additionally, since new classification algorithms are developed rapidly it is very essential to evaluate their performance and sensitivity in different environs using various types of remote sensing datasets and high quality samples.If a comprehensive assessment of algorithms on various kinds of environment types for F 1 score attained for every land cover type.For Random Forest both data group A and C provides the best classification accuracy in terms of overall accuracy (average OA = 95.7% and 96.9% respectively).Higher F 1 score index values are obtained for all land cover types ranging between 0.94 and 1 (Table2).Group B indicates higher overall classification land cover types.Category B indicates poor lower overall classification accuracy (average OA = 57.9%)andlowerF 1 score index values are obtained for dense forest, moderate forest and bare soil covers ranging from 0.07 to 0.68 (Table2).Figure 2. Comparison of the overall classification accuracy achieved on different data categories (Table 1), for MLC, NN, SVM and RF classifiers based on the validation samples.Table 2. F 1 score accuracy comparison of land cover classification results for different data groups achieved for the tested classification algorithms; (a) SVM (b) RF (c) NN (d) MLC.Results are based on the validation dataset.DOI: 10.4236/ars.2018.7200454 Advances in Remote Sensing

Table 1
) and classifiers.The results are based on validation samples.D. Deus DOI: 10.4236/ars.2018.7200455 Advances in Remote Sensing

Table 3 .
[1]est statistic results for the comparison of the classifiers.The comparison was done based on the overall classification accuracy attained for each data category.On the integration of SAR and Landsat data all classifiers indicate good performance, however, SVM and RF has the best performance in relation to NN and MLC at 95% confidence interval.Based on previous studies, parametric classifiers like MLC are not worthy when using multi-source remote sensing data.The superior performance of SVM and RF compared to NN could be due to the fact that SVM and RF has the potential to handle high dimensional data[1].