^{1}

^{*}

^{2}

^{3}

^{4}

^{5}

The eddy covariance technique is an accurate and direct tool to measure the Net Ecosystem Exchange (NEE) of carbon dioxide. However, sometimes conditions are not amenable to measurements using this technique. Thus, different methods have been developed to allow gap-filling and quality assessment of eddy covariance data sets. In this study first, two different Artificial Neural Networks (ANNs) approaches, the Multi-layer Perceptron (MLP) trained by the Back-Propagation (BP) algorithm, and the Radial Basis Function (RBF), were used to fill missing NEE data measured above rain-fed maize at the University of Nebraska-Lincoln Agricultural Research and Development Center near Mead, Nebraska. The gap-filled data were then compared by different statistical indices to gap-filled data obtained with the technique suggested by Suyker and Verma in 2005 [S&V method], and the ANN approach presented by Papale in 2003. The results showed that the RBF network was able to find better fits for missing values compared to the MLP (BP) network and S&V method. In addition, unlike the S&V method, which depends on different gap-filling procedures over the year; the structure of RBF and MLP (BP) networks was constant. However, data analysis indicated Papale’s approach gave better fits than the RBF and MLP (BP) methods. Thus, based on this work, Papale’s approach is the best method to estimate the missing data; though the applied statistical indices, which were used for model evaluation, show little difference between Papale’s approach and the RBF and MLP (BP).

The eddy covariance technique is one of the most accurate and direct tools to help us measure the carbon exchange between the surface and the atmosphere in various ecosystems.

Most often eddy covariance data quality has problems, as a result of instrument malfunctions, power outage, extreme weather conditions, and insufficient turbulent mixing [

Stable conditions and low turbulence are factors that result in under-estimation of daily NEE. Since these conditions often occur at night the Net Ecosystem Exchange (NEE) values are removed when the measured wind speeds are less than a threshold friction velocity (u*) [

To fill missing data, various methods are applied such as non-linear regressions, dual unscented Kalman filter, semi-parametric models, terrestrial biosphere models, and artificial neural networks [

Although intensive efforts in gap-filling methods have already been made, there is a necessity for conducting more research in order to establish the accuracy and reliability of the techniques based on variability in plant, climate, and particularly, the quality and quantity of the data set.

In this study, first, we investigated the function of two different ANN methods known as the Multi-layer Perceptron (MLP) trained by the Back-propagation (BP) algorithm and the Radial Basis Function (RBF) in order to fill the missing NEE data for rain-fed maize. Then, the gap-filled data, using mentioned methods, were compared with the technique suggested by Verma & Suyker (V&S) [

1) Artificial neural networks:

Artificial Neural Networks (ANNs) are dynamic systems, which can compute and identify the relationships among input and output data to obtain estimation with the best approximation [

Today, ANNs are powerful tools that can approximate many complex phenomena. They were trained to perform pattern recognition, identification, classification, and control systems. ANNs are particularly suited to fill data gaps for NEE flux using other sources of data [

2) Multi-layer perceptron network:

Multi-layer Perceptron (MLP) Networks are feed-forward neural networks containing input layers, an output layer, and one or more hidden layers. An MLP is a fully connected network (

The best systematic method for training MLP networks is the Steepest Descent Back-propagation, organizing two main paths, forward and backward (Safa et al., 2011). In the forward path, the input vector is applied through the MLP network and its effects, via the hidden layer, are distributed onto the output layer without any change in the network parameters. In the backward path, unlike the first one, all network parameters change and are adjusted by the error correction rule. An error vector is formed in the output layer and is taken into account by the difference between the actual (observed) and estimated values. The error values from the output layer are distributed in the entire network. This process is continued until the amount of error becomes as low as possible and stability is observed in network parameters.

Equations (1) and (2) respectively, show the adjustment process in the output and hidden layers (Kumar et al., 2002):

w ( N + 1 ) = w ( N ) − η δ ϕ (1)

w ( N + 1 ) = w ( N ) + η χ ∑ q = 1 r δ q (2)

where; w is weight, N the number of iteration, χ input value, η learning rate, ϕ output, and δ is defined as 2 ε q ∂ ϕ / ∂ I , I the sum of the weighted inputs, q neuron index of the output layer, and ε q error signal.

The learning rate ( η ) determines the step span on the activation function over each repetition of network parameters optimization. If lower values of the learning rate are selected, the changes in the network parameters will be smaller after each repetition, a case that will help to smooth the movement path of parameters toward the optimum quantities and will slow down the learning process. Inversely, when the learning rate is increased, although the learning speed is increased too, large changes are made from one repetition to the next, which occasionally will bring instability in the network situation generally referred to as divergent network parameters [

The momentum parameters can help to avoid this problem. Momentum is defined as the amount of inertia that is increased in each network parameter. It is utilized to improve the learning rate and prevents instability in the network [

Δ w ( N + 1 ) = − η δ ϕ + μ Δ w ( N ) (3)

where; μ is momentum coefficient and Δ w ( N + 1 ) the change of weight during N to N + 1 learning cycles.

One of the other network parameters is epoch. One entire set of inputs which is given to the network at each learning cycle is called an epoch.

3) Radial basis function networks:

A Radial Basis Function (RBF) network is a fully inter-connected feed-forward structure composed of three layers (input, hidden, and output). The activation function in hidden neurons is a Gaussian exponential whereas, in the output layer, it is a linear function. The Gaussian function consists of two main parameters, center (c) and width ( σ ). In each neuron in the hidden layer, the Euclidean (radial) distance (x − c) is computed where x is input vector, and c is the prototype vector, which represents the center of RBF.

The output value for the hidden layer (

h j p = R j p / ∑ j = 1 L R j p ( p = 1 , 2 , ⋯ , P ) , ( j = 1 , 2 , ⋯ , M )

R j p = exp [ − 1 2 σ j ∑ k = 1 N ( x k p − c j k ) 2 ]

Furthermore, the output of the output layer is computed by:

y i p = w o j ∑ j = 1 L w i j h j p

where, N, L,and M are the number of units in input, hidden, and output layers; respectively, and P is the number of input patterns, R j p is the hidden layer activation function, h j p is the normalization of the hidden layer activation function.

RBF networks are trained using supervised and unsupervised learning [

c i j = 1 M j ∑ p ∈ θ j x i p (4)

where, M j is the number of sample nodes in the sample subset, θ j . Second, the width ( σ j ) of the basis function ( c j ) is determined using a P-nearest neighbor method which is expressed by the mean distance between the centers of the basis function and sample modes in the sample subset:

σ j = 1 M j ∑ p ∈ θ j ∑ k = 1 n ( x k p − c j k ) 2 (5)

In the supervised phase, the weights ( θ j ) between hidden layer neurons and the output layer neurons are determined according to the least-squares principle.

E s = 1 2 ∑ p = 1 P ∑ k = 1 M ( y k p − ∑ j = 1 L w j k h j p ) (6)

where, E s is the function of θ j and is gotten by least-squares principle, y k p is observed output.

RBF networks in comparison with the Back-propagation (BP) networks have some advantages and disadvantages. For instance, they train faster than BP networks with less sensitivity to local minima [

One of the major disadvantages of RBF networks is some pre-selection of input data in the hidden neurons, due to the initial learning method during the unsupervised clustering phase, which leads to losing some information contributing to the output neurons [

4) Data screening and gap filling procedures using Verma& Suyker’s method:

NEE data screening procedures proposed by Verma and Suyker contain four steps: stationarity (NEE data are accepted by criteria discussed by [

5) Papale’sapproach:

Recently, AmeriFlux scientists at level 4 (Data obtained from level 3 products. Data are u* filtered and gap–filled using different methods) have filled in the missing NEE values using the method proposed by [

6) Site description and measurements:

The selected field (Site 3) is located at the University of Nebraska-Lincoln Agricultural Research and Development Center near Mead, NE. According to the Köppen climate classification, Mead has a temperate climate; the mean annual precipitation and temperature (1968-2011) are 719.6 mm and 9.9˚C, respectively. Moreover, about 71% at annual precipitation occurs over the growing season (May-October). The soils are deep, silty, clay loams.

Site 3 (41˚10'46.8''N, 96˚26'22.7''W, 362 m above mean sea level) is not irrigated and during 2001-2009 was planted to a maize-soybean rotation. In this research, the maize years 2001, 2003, 2005, 2007, and 2009 were studied. The ANN method presented by [

Hourly NEE and the input data were selected from day of year (DOY) 169 to 244 for selected years. To measure NEE flux, an omnidirectional 3D sonic anemometer (R3, Gill Instruments Ltd., Lymington, UK) and a closed path CO_{2}/H_{2}O gas analyzer (LI 6262, LI-Cor Inc., Lincoln, NE) were utilized. For sufficient fetch, the sensors were installed 3.0 m above the ground when the canopy was shorter than 1.0 m height, and then moved to 6.0 m height for the reminder of season. Air temperature and vapor pressure at 3.0 m, and 6.0 m were measured with a HMP 35 C (Viasala, Helsinki, Finland). Soil temperature at 0.1 m depth was measured with platinum RTD (Omega Engineering, Stamford, CT). Photosynthetically active radiation was measured with a quantum sensor (LI 190 SA, LI-Cor Inc., Lincoln, NE), and net radiation with a net radiometer (Q*6, REB systems, Seattle, WA). A tipping bucket, TE 525 (Campbell Scientific, Logan, UT), and a cup anemometer (010 C Met One, Instruments, Grants Pass, OR) were used to measure precipitation and wind speed. Soil water content was measured using a ML2x ThetaProbe (Delta-T, Burwell, Cambridge, UK). LAI was determined using a destructive sampling. A LI-3100 (LI-Cor Inc., Lincoln, NE) leaf area meter was used to measure sampled leaves. The period of LAI samplings was generally every 7 to 10 days with due attention that one sampling should be timed to correspond to maximum leaf area index which usually occurs at tasseling.

7) Data selection:

To design an ANN, the optimal selection of input elements is a key factor. The understanding of the governing physical and biological laws among input and output elements plays an important role in ANN modeling. Some of the important physiological, meteorological, and edaphic elements that have an effect on NEE flux include leaf area index (LAI), [m^{2}/m^{2}]; soil water content (SWC), in 10 and 25 cm depths; soil temperature (Ts), [˚C], in 10 cm depth; air temperature (Ta), [˚C]; vapor pressure deficit (VPD), [kPa]; wind speed (WS), [m/s]; irrigation and precipitation (P), [mm]; net radiation (Rn), [W/m^{2}]; and the fraction of incoming photosynthetically active radiation (PAR) absorbed by the canopy (fPAR). Furthermore, local time of day, each hour equals 0.0417 (DTime), was added to input elements in order to improve the training of ANN.

8)Gap filling procedures:

Hourly data for twelve input elements accompanied by hourly NEE as output were used in the data matrix architecture (

In the matrix, the rows were the calendar years, and the columns were input elements plus output values. After making the data matrix, one year-data (20%) was extracted to make a test file and the others (80%) were applied to the learning file. The year 2005 was selected for the test file. Corresponding to the climatic data records, 2005 had relatively moderate conditions based on precipitation and temperature over the growing season in comparison with the other years. The validation file was developed using the data matrix with NEE observed (screened) values that belonged to the year 2005. This file was not used for training processes. The observed (screened) are data, which after quality control, are recognized to be filled with the new values. In the architecture of MLP networks, three hidden layers networks with the Normalized Cumulative Delta (NCD) rule were used. According to NCD, the adjusted weights at the end of each epoch are stored and the learning rule is independent of the epoch size. The applied activation functions in hidden layers were Tangent Hyperbolic, which ranges in values between [−1, +1]. In the output layer, a linear function was used.

The optimum network response was obtained when the epoch chosen was one. The input data were normalized to the interval [−1, +1] which is idiomatically named Bipolar. To design the RBF network using the [

9) Model evaluation:

Apart from the usual quantitative measures used for the evaluating performance of a model, our study applied the following statistical indices in order to select the best architecture of networks based on the difference between the observed and estimated values.

Root Mean Square Error (RMSE):

RMSE = 1 n ∑ d = 1 n ( x d − x ^ d ) 2

Mean Absolute Error (MAE):

MAE = 1 n ∑ d = 1 n | x d − x ^ d |

where x d is the estimated value, x ^ d the observed value with “n” observations.

Elements DOY | Year | DTime | LAI^{(m}^{2}^{/m}^{2}^{)} | SWC (10 cm) | SWC (25 cm) | Ts (10 cm)^{(}^{˚}^{C)} | Ta^{(}^{˚}^{C)} | VPD^{kKPa}^{)} | P^{(mm)} | WS^{(m/s)} | Rn^{(W/m2)} | APAR pct^{(}^{%)} | NEE Flux^{(mg/m2/s)} (Output) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

169 | |||||||||||||

170 | |||||||||||||

171 | |||||||||||||

… | |||||||||||||

242 | |||||||||||||

243 |

1) Results of the comparison of NEE flux estimation using the MLP (BP),RBF,and Verma& Suyker’s methods:

After making the data matrix, the training procedures with MLP (BP) and RBF using the initial network parameters for the learning and test file was initiated and continued to achieve the best network parameters based on model evaluation methods. The values of the network parameters used are listed in

The quality control of the NEE values for the selected time period in 2005 using Verma & Suyker’s approach indicated a number of 468 observed (screened) values that are necessary to be filled by the new data. This dataset formed the validation file. _{max} (|Maximum Error Value – Minimum Error Value|) also demonstrate that the smallest values were observed for the RBF method. Schmidt [

Figures 4(a)-(c) show that there is no specific pattern between the estimated and observed (screened) NEE values. In other word, a significant difference is observed between the estimated and observed (screened). However,

Figures 5(a)-(c) depict the dispersion of differences between the estimated and observed (screened) NEE (error) quantities. They indicate no remarkable difference between observed (screened) and estimated NEE for the three applied models though, the e_{max} has the lowest value for the RBF network (

ANNs | # PE in | # PE in | # PE in | # PE in | Learning Coefficient in Momentum | |||
---|---|---|---|---|---|---|---|---|

input | hidden 1 | hidden 2 | output | input | hidden 1 | hidden 2 | output | |

MLP (BP) RBF | 12 12 | 24 50 | 6 34 | 1 1 | 0.300 0.300 | 0.200 0.250 | 0.150 0.150 | 0.400 0.400 |

Models | RMSE (mg·m^{−2}·s^{−1}) | MAE (mg·m^{−2}·s^{−1}) | e_{max} (mg·m^{−2}·s^{−1}) |
---|---|---|---|

MLP (BP) RBF V&S | 0.2989 0.2970 0.2978 | 0.1867 0.1805 0.1825 | 3.0342 2.9360 3.0263 |

e max = | Maximum Error Value − Minimum Error Value | .

Figures 6(a)-(c) show that there is no consistency between observed (screened) and estimated NEE over the specified time series among the models. Data analysis indicated that the entire gap-filled data belonged to the nighttime hours. Generally, the NEE values must be positive at this moment whereas, they were negative (according to the AmeriFlux network). Moreover, the recorded values mostly contained very large amounts. These irregularities and disturbances in recorded data, lead to a striking disagreement between the observed (screened) and estimated data.

Figures 7(a)-(c) illustrate a short segment (5.5 days) of the NEE variation with the highest reported gap data in 2005 for observed (quality-controlled), estimated, and observed (screened) data using three applied methods.

One of the major advantages of the gap-filling procedures using RBF and MLP is that a single computation method is applied throughout a year whereas; Verma & Suyker’s gap-filled producers are classified into three time periods. For each of them, the computation method of missing values is different. The study by [

Overall, this study suggests that the application of RBF method to fill the gap data in comparison with Verma & Suyker’s gap-filled producers is provide values that are more accurate.

2)Results of the comparison of NEE flux estimation using the MLP (BP),RBF,and Papalemethods:

Papale’s approach has been applied to fill the gap data for the years of 2001 to 2005 by AmeriFlux network. 2005 was selected for making the test data file due to the relatively moderate conditions based on precipitation and temperature over the growing season in comparison with the other years. It is clear that the other years (2001 & 2003) were used to make the data matrix for learning file. Eventually, the networks training procedures using the MLP (BP), RBF, and Papale’s methods were accomplished. The network parameters for these methods are shown in

The quality control of the NEE values for the selected time period in 2005 using Papale’s approach indicated a number of 295 observed (screened) values that are necessary to be filled by the new data. This dataset formed the validation file. The model evaluation results according to the error analysis for the validated file are shown in _{max} belong to Papale’s approach.

The relation between observed (screened) and estimated data is shown in Figures 8(a)-(c). The stronger relationship based on pattern of NEE change was observed in

The difference between observed (screened) and estimated NEE is depicted in Figures 9(a)-(c). The lowest dispersion belongs to Papale’s method in _{max} has the least value for this method and however, the e_{max} for the RBF network was very close to it.

Figures 10(a)-(c) depict the comparison of observed (screened) and estimated NEE using the applied three methods. The trend of estimated NEE by Papale and the MLP (BP) approaches have more compatibility with observed (screened) values compared to the RBF method. On the other hand, the fluctuation of estimated values is less than the others in

ANNs | # PE in | # PE in | # PE in | # PE in | Learning Coefficient in Momentum | |||
---|---|---|---|---|---|---|---|---|

input | hidden 1 | hidden 2 | output | input | hidden 1 | hidden 2 | output | |

MLP (BP) RBF | 12 12 | 28 50 | 2 31 | 1 1 | 0.300 0.300 | 0.200 0.250 | 0.150 0.150 | 0.400 0.400 |

Models | RMSE (mg·m^{−2}·s^{−1}) | MAE (mg·m^{−2}·s^{−1}) | e_{max} (mg·m^{−2}·s^{−1}) |
---|---|---|---|

MLP (BP) RBF Papale | 0.3881 0.3762 0.3285 | 0.2816 0.2635 0.2004 | 2.7724 2.5619 2.5301 |

e max = | Maximum Error Value − Minimum Error Value | .

Figures 11(a)-(c) illustrate the observed (quality-controlled), estimated, and observed (screened) values for a short segment (5.5 days) of the NEE variation in 2005, using the applied methods. The representative time period is exactly the same as

There may be several reasons that support Papale’s approach compared to our methods, such as input variables and network characteristics.

Input variables:According to [_{2} flux dataset used by Papale’s approach contained 16 European sites (each site consisted of about 17,500 samples) within different climate zones and ecological parameters. Thus, using the larger data set (more samples) in order to train the network by Papale’s approach promote the efficiency of his method compared to the other applied methods in this study.

Network characteristics: The dissimilarity in network architecture and parameters are influential factors on the model performance; though there is no information regarding the network architecture and parameters in the references.

Eddy covariance method is one of the best approaches to measure net ecosystem exchange flux above plant canopies. However, data collection using this method is prone to gaps. Today, several techniques are used to fill the missing data. In this study, the capability of the Multi-layer Perceptron (MLP) network trained by the Back-Propagation (BP) algorithm and the Radial Basis Function (RBF) network are compared with each other. Their results then are compared to Verma and Suyker’s (V&S) method for the same time period.

The results confirmed the RBF network succeeded in the best fit for observed (screened) values based on statistical tests. Moreover, using a single method for gap-filling over a year was the advantage of the application of Artificial Neural Networks (ANNs) compared to the V&S approach.

In the second part, the preference of RBF and MLP (BP) were compared to Papale’s method, which fundamentally was also an ANN-based on MLP (BP). Results indicated that there was no striking difference among the three methods. Nevertheless, according to statistical indices, the Papale method was the best. Several reasons could be given for attaining the best estimation for NEE by Papale’s method. One of them is the structure of data matrices, including both qualities and quantities. The network architecture and parameters are the other effective factors to this achievement.

Overall, the results show that ANNs, as a technique, is able to estimate the missing NEE data with good accuracy and efficiency.

The authors declare no conflicts of interest regarding the publication of this paper.

Safa, B., Arkebauer, T.J., Zhu, Q.M., Suyker, A. and Irmak, S. (2021) Gap Filling of Net Ecosystem CO_{2} Exchange (NEE) above Rain-Fed Maize Using Artificial Neural Networks (ANNs). Journal of Software Engineering and Applications, 14, 150-171. https://doi.org/10.4236/jsea.2021.145010