Interpolation of Generalized Functions Using Artificial Neural Networks

In this paper we employ artificial neural networks for predictive approximation of generalized functions having crucial applications in different areas of science including mechanical and chemical engineering, signal processing, information transfer, telecommunications, finance, etc. Results of numerical analysis are discussed. It is shown that the known Gibb’s phenomenon does not occur.


Introduction: Main Definitions, Representations and Relations between Known Distributions
Generalized functions are used in modeling and analysis of applied systems in various areas of science including engineering, finance, control theory, etc. Practically every object or phenomenon containing discontinuity, switches or localized character, in principle, can be described in terms of generalized functions or distributions.The theory of generalized functions is developed by Sergei Sobolev (1930s) and Laurent Schwartz (1940s) (for details see the monographs [1] [2] [3] and the references therein).Nevertheless, George Green and Oliver Heaviside used generalized functions in their research much earlier.The Dirac's δ function defined by ( ) is used in the Green's representation formula for the general solution of nonhomogeneous boundary value problems.Later in 1930s, Paul Dirac systematically used the δ function to describe a point charge localized at a given point.In practical analysis, the definition of the Dirac's δ(1) must be supplemented by for arbitrary continuous function f.On the other hand, Heaviside used the θ function given by ( ) to extend the notion of the Laplace integral transform in telegraphic communications.At this, the value θ(0) depends on particular problem and can be one of 0, 1 or 0:5.
According to the definition, the Dirac's and Heaviside's functions are related by In other words, θ is the antiderivative of δ in the sense of generalized functions.On the other hand, the function max {x, 0} is the antiderivative of θ, therefore which direct follows from the second equality in (2) with f(x) = x.
There exists a linear relation between the Heaviside's generalized function and the sign function defined by: ( ) ( ) Evidently, here the value θ(0) = 0.5 is considered.However, it is possible to write such a formula with θ(0) = 0 or θ(0) = 1.
Other known generalized functions can be defined through θ.For instance, the characteristic function defined by The rectangular function defined by ( ) can be expressed in terms of θ as follows: ( ) As above, here the value θ(0) = 0.5 is considered as well.
Another well-known generalized function is defined through the convolution where * denotes the convolution operation.This function is called ramp function and has many applications in engineering (it is used in the so-called half-wave rectification, which is used to convert alternating current into direct current by allowing only positive voltages), artificial neural networks (it serves as an activation function), finance, statistics, fluid mechanics, etc.
According to the definition of the Heaviside function, the rump function can be represented also as

Approximation of Main Generalized Functions by Means of Locally Measurable Functions
The theory of generalized functions is a very well developed mathematics subject crucial for rigorous analysis of many applied systems.Nevertheless, their rigorous definitions are completely useful in numerical analysis, because they are even not proper functions.In numerical analysis proper functional approximations of the generalized functions are used instant.
In practice, the approximation of generalized functions is based on construction of a sequence f n of measurable functions giving the desired generalized function in limit when n → ∞.For instance, the sequence ( ) ( ) which is also called Gauss kernel, tend to the Dirac's generalized function when n → ∞.The sequence δ n is called δ-like sequence.Several other δ-like sequences can be found in literature.Examples include ( ) which is also called Dirichlet kernel, and  Journal of Computer and Communications which is also called Fejér kernel.Note that for all mentioned kernels, ( ) i.e. they are locally measurable functions.Taking into account (4), similar θ-like sequences can be constructed for approximating Heaviside's θ.For instance, ( ) ( ) often referred to as logistic function, ( ) ( ) Similar sequences can be constructed for the functions sign, χ, rect and R above.For example, ( ) can be viewed as rect-like sequences.
On the other hand, the expression ( ) can be used as approximation to the ramp function.

Approximation of Generalized Functions Using Artificial Neural Networks
Generalized this paper we show through numerical experiments that artificial neural networks can provide a very fast and efficient approximation for generalized functions using any of the approximate formula above.There is a huge body of references devoted to the theory and implementation of artificial neural networks for approximation of functions.We refer to [4]

[ 5 ]−
[6] [7][8] and the references therein.The neural network providing approximation consists of an input layer, a hidden layer and output layer.The quadratic error of approximation where f is the original function and f app is its approximation.Moreover, in all examples below the θ-like sequence (5) considered.Other sequences can be applied exactly in the same way.The learning rate is always fixed to 10 −3 for simplicity.Approximation of the rect function for different number of nodes is presented in Figure1.A better approximation with less error can be obtained by increasing the learning rate or the number of nodes.The error is plotted on Figure2, from which it is obvious that the least error is 4 ~10 ε It is evident from Figure 3 that the known Gibbs phenomenon does not occur here [9].