Function Approximation Using Robust Radial Basis Function Networks

Oleg Rudenko, Oleksandr Bezsonov

.

DOI: 10.4236/jilsa.2011.31003                

Abstract

Resistant training in radial basis function (RBF) networks is the topic of this paper. In this paper, one modification of Gauss-Newton training algorithm based on the theory of robust regression for dealing with outliers in the framework of function approximation, system identification and control is proposed. This modification combines the numerical ro- bustness of a particular class of non-quadratic estimators known as M-estimators in Statistics and dead-zone. The al- gorithms is tested on some examples, and the results show that the proposed algorithm not only eliminates the influence of the outliers but has better convergence rate then the standard Gauss-Newton algorithm.

Keywords

Neural Network, Robust Training, Basis Function, Dead Zone

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	B. Kosko, “Neural Network for Signal Processing,” Pren-tice-Hall Inc., New York, 1992.
[2]	S. Haykin, “Neural Net-works. A Comprehensive Founda- tion,” 2nd Edition, Prentice Hall Inc., New York, 1999.
[3]	C. M. Bishop, “Neural Net-work for Pattern Recognition,” Clarendon Press, Oxford, 1995.
[4]	H. Wang, G. P. Liu, C. J. Harris and M. Brown, “Advanced Adaptive Control,” Pergamon, Oxford, 1995.
[5]	R. Hecht-Nielsen, “Kolmogorov’s Mapping Neural Net- works Existence Theorem,” First IEEE International Conference on Neural Networks, San Diego, Vol. 3, 1987, pp. 11-14.
[6]	G. Cybenko, “Approximation by Superpositions of a Sigmoidal Function,” Mathematics of Control, Signals and Systems, Vol. 2, No. 4, 1989, pp. 303-314. doi:10. 1007/BF02551274
[7]	T. Poggio and F. Girosi, “Networks for Approximation and Learning,” Proceeding of the IEEE, Vol. 78, No. 9, 1990, pp. 1481-1497. doi:10.1109/5.58326
[8]	J. Park and I. W. Sandberg, “Universal Approximation Using Radial-Basis-Function Network,” Neural Compu- tation, Vol. 3, No. 2, 1991, pp. 246-257. doi:10.1162/ ne-co.1991.3.2.246
[9]	C. C. de Wit and J. Carrillo, “A Modified EW-RLS Algorithms for Systems with Bounded Disturbances,” Automatica, Vol. 26, No. 3, 1990, pp. 599-606. doi:10.1016/ 0005-1098(90)90032-D
[10]	P. J. Huber, “Robust Statistics,” John Wiley, New York, 1981. doi:10.1002/0471725250
[11]	R. E. Frank, M. Hampel, M. Rohchetti and W. A. Stanel, “Robust Statistics: The Approach Based on Influence Functions,” John Wiley & Sons Inc., Hoboken, 1986.
[12]	C. C. Chang, J. T. Jeng and P. T. Lin, “Annealing Robust Radial Basis Function Networks for Function Approxi- mation with Outliers,” Neu-rocomputing, Vol. 56, 2004, pp. 123-139. doi:10.1016/S0925-2312(03)00436-3
[13]	S.-C. Chan and Y.-X. Zou, “A Recursive Least M-Esimate Algorithm for Robust Filtering in Impulsive Noise: Fast Algorithm and Convergence Performance Analysis,” IEEE Transactions on Signal Processing, Vol. 52, No. 4, 2004, pp. 975-991. doi:10.1109/TSP.2004.823496
[14]	D. S. Pham and A. M. Zoubir, “A Sequential Algorithm for Robust Parameter Estima-tion,” IEEE Signal Proce- ssing Letters, Vol. 12, No. 1, 2005, pp. 21-24. doi:10.11 09/LSP.2004.839689
[15]	J. Ni and Q. Soug, “Pruning Based Robust Backpropaga- tion Training Al-gorithm for RBF Network Training Con- troller,” Intelligent and Robotic Systems, Vol. 48, No. 3, 2007, pp. 375-396. doi:10.1007/s10846-006-9093-x
[16]	G. Deng, “Sequential and Adaptive Learning Algorithms for M-Estimation,” EURASIP Journal on Advances in Signal Processing, Vol. 2008, 2008, ID 459586.
[17]	C.-C. Lee, Y.-C. Chiang, C.-Y. Shin and C.-L. Tsai, “Noisy Time Series Prediction Using M-Estimator Based Robust Radial Basis Function Network with Growing and Pruning Techniques,” Expert Systems with Applications, Vol. 36, No. 3, 2008, pp. 4717-4724. doi:10.1016/j.eswa. 2008.06.017
[18]	E. Fogel and Y. E. Huang, “On the Value of Information in System Identification Bounded-Noise Case,” Auto- matica, Vol. 18, No. 2, 1982, pp. 229-238. doi:10.1016/ 0005-1098(82)90110-8
[19]	R. Lozano-Leal and R. Ortega, “Reformulation of the Parameter Identification Problem for Systems with Bounded Disturbances,” Automatica, Vol. 23, No. 2, 1987, pp.247-251. doi:10.1016/0005-1098(87)90100-2
[20]	J. Chambers and A. Alvonitis, “A Robust Mixed-Norm Adaptive Filter Algorithm,” IEEE Signal Proceeding Letters, Vol. 4, No. 22, 1997, pp. 46-48. doi:10.1109/97. 554469
[21]	Y. Zou, S. C. Chan and T. S. Ng, “A Recursive Least M-Estimate (RLM) Adaptive Filter for Robust Filtering in Impulse Noise,” IEEE Signal Proceeding Letters, Vol. 7, No. 11, 2000, pp. 324-326. doi:10.1109/97.873571
[22]	P. W. Holland and R. E. Welsh, “Robust Regression Us- ing Iteratively Reweighted Least Squares,” Communications in Statistics-Theory Mathematics, Vol. A6, 1977, pp. 813-827. doi:10.1080/03610927708827533
[23]	S. Geman and D. McClure, “Statistical Methods for Tomographic Image Recon-struction,” Bulletin of the International Statistical Institut, Vol. L2, No. 4, 1987, pp. 4-5.
[24]	K. S. Narendra and K. Parthasa-rathy, “Identification and Control of Dynamical Systems Using Neural Networks,” IEEE Transactions on Neural Networks, Vol. 1, No. 1, 1990, pp. 4-26. doi:10.1109/72.80202