^{1}

^{2}

^{3}

Artificial Neural Network (ANN) equalizers have been successfully applied to mitigate Inter symbolic Interference (ISI) due to distortions introduced by linear or nonlinear communication channels. The ANN architecture is chosen according to the type of ISI produced by fixed, fast or slow fading channels. In this work, we propose a combination of two techniques in order to minimize ISI yield by fast fading channels, i.e., pulse shape filtering and ANN equalizer. Levenberg-Marquardt algorithm is used to update the synaptic weights of an ANN comprise only by two recurrent perceptrons. The proposed system outperformed more complex structures such as those based on Kalman filtering approach.

It is well known that the mobile communication system performance is degraded by thermal noise, Intersymbolic Interference (ISI) and fading. Assuming that the thermal noise is Additive White Gaussian Noise (AWGN channel), its effects can be minimized using matched filter at the receiver and equalizers can be employed to combat ISI effects and fading. Most papers in the scientific literature deal with time-invariant channels, but nowadays the majority of channels are time-variant, which diminish system performance and their effects are difficult to mitigate. Recent works [1-3] analyse systems involving slow fading channels but do not propose robust solutions to fast fading.

Artificial Neural Network (ANN) has been largely used in equalization problems where equalizers based on perceptrons are applied to improve performance of some nonlinear channels [4,5]. In the present paper, two independent recurrent perceptrons are applied, separating the processing of real and imaginary parts of the signal, providing a more accurate solution by avoiding the approximation

The Recurrent Neural Network (RNN) proposed in [

To reduce the unstable behavior of the GN algorithm, in this work we propose to use the Levenberg-Marquardt (LM) approach, a well-known modification of the GN algorithm, which includes the convergence guarantee of Gradient techniques. However, the replacement of algorithm is not enough to solve the fast fading equalization problem, and other subsystems must be added to the transmitter and receiver front-ends in order to improve signal quality, before being processed by the equalizer (

In practical communications systems, it is necessary to implement a scheme capable of mitigate signal distortions caused by ISI and AWGN in fading channels. In this work the proposed scheme is composed by an equalizer and a pulse shaping filter that together can deal better with distortions in 4-QAM signals. The equalizer is a RNN with two perceptrons, each one as proposed in [

The proposed RNN equalizer is equivalent to a Decision Feedback Equalizer (DFE) based on perceptrons and implemented with parameters m = 3, n = 2 and d = 2 [

The activation function chosen for the DFE-LM training is the hyperbolic tangent function, whereas the hard decision sign function is implemented during testing. The present work does not use the well-known approximation

the real and imaginary parts of 4-QAM symbols.

At the transmitter, it changes the signal shape by using a raised cosine filter which limits the pulse spectral components. The roll-off factor defines the transmitted signal bandwidth. Roll-off factor of 0.2, which was chosen by trial and error, provided the best system perform.

On the order hand, the Pulse Shaping filter limits the amount of the noise spectrum that is passed on to next stages at the receiver and correlates the transmitted and received signals. In this paper were consider two samples per symbol, the minimum amount to allow the pulse shaping filter to work properly and make possible to reach higher transmission rates.

A GN algorithm modification, proposed by LevenbergMarquardt, is implemented in this work to update the synaptic weights during the DFE-LM training. In the LM algorithm the synaptic weights are updated according to Equation (1).

The LM differs from GN algorithm by the λI term.

The new term improves the convergence stability of the GN algorithm. The λ factor must increase in Equation (1) as the error e(k) increases, making the algorithm behaves like a Gradient algorithm. If the error decreases for each update step, then the λ factor must decrease in Equation (1) as well, leading to a GN type solution. There is a trade-off between convergence assurance of the Gradient algorithm and fast convergence rate of the GN algorithm.

The error is a function of several local minima. Sometimes, after the training phase, the algorithm converges to a set of weights that does not satisfy an acceptable Bit Error Rate (BER). To avoid this possibility, we decided to introduce, in the algorithm, a BER evaluation before start the test phase. If the BER is greater than a threshold, then the algorithm initial value is changed and the training phase is run again. This approach enables the algorithm to escape from local minimum solutions and allow it to carry on searching for a better minimum, capable of satisfying the best BER requirement.

The system performance was evaluated over different time-variant channels, modeled by the transfer function in Equation (2).

When

The time-variant coefficients are independently generated by passing white noise signal through a low pass second-order Butterworth filter, with normalized cut-off frequency at 0.1 [

It may seem unfair compare systems with different structures. The pulse shaping filters could improve the DFE-EKF and DFE-UKF results, but due to its high computational cost, they do not seem to be suitable for mitigation of fast fading channel ISI in practical applications. Furthermore, the GN algorithm convergence issues cannot be solved by adding the pulse shaping filter.

Finally,

LM over the channel model (Equation (2)) with coefficient standard deviation 0.1, 0.3 and 0.5, keeping the same simulations conditions described before. The BER performances are acceptable even for channels with standard deviation of 0.5 (fastest fading) and SNR of 6dB. This last result shows that the proposed equalization system is a promising alternative to deal with a severe fast fading channel ISI.

Both of Figures 2 and 3 show the recurrent neural structures performance applied to the equalization problem. From the literature [

A RNN based on the DFE structure was proposed to mitigate 4-QAM signal distortions over fading channels. A pulse shaping filter was added to overcome the equalizer limitations in this task. The synaptic weights were updated during the training phase by the LM algorithm. The approach was evaluated in traditional time-varying channels mentioned in the literature, and the signal was submitted to different levels of fading. The proposed structure has been able to deal with severe time-varying conditions and outperform previous works [1,2] in low SNR.