An Autonomous Incremental Learning Algorithm for Radial Basis Function Networks

Seiichi Ozawa, Toshihisa Tabuchi, Sho Nakasaka, Asim Roy

.

DOI: 10.4236/jilsa.2010.24021                

Abstract

In this paper, an incremental learning model called Resource Allocating Network with Long-Term Memory (RAN-LTM) is extended such that the learning is conducted with some autonomy for the following functions: 1) data collection for initial learning, 2) data normalization, 3) addition of radial basis functions (RBFs), and 4) determination of RBF cen-ters and widths. The proposed learning algorithm called Autonomous Learning algorithm for Resource Allocating Network (AL-RAN) is divided into the two learning phases: initial learning phase and incremental learning phase. And the former is further divided into the autonomous data collection and the initial network learning. In the initial learning phase, training data are first collected until the class separability is converged or has a significant dif-ference between normalized and unnormalized data. Then, an initial structure of AL-RAN is autonomously determined by selecting a moderate number of RBF centers from the collected data and by defining as large RBF widths as possible within a proper range. After the initial learning, the incremental learning of AL-RAN is conducted in a sequential way whenever a new training data is given. In the experiments, we evaluate AL-RAN using five benchmark data sets. From the experimental results, we confirm that the above autonomous functions work well and the efficiency in terms of network structure and learning time is improved without sacrificing the recognition accuracy as compared with the previous version of AL-RAN.

Keywords

Autonomous Learning, Incremental Learning, Radial Basis Function Network, Pattern Recognition

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	R. Sun and T. Peterson, “Autonomous Learning of Sequential Tasks: Experiments and Analyses,” IEEE Transaction on Neural Networks, Vol. 9, No. 6, 1998, pp. 1217-1234.
[2]	J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur and E. Thelen, “Autonomous Mental Development by Ro-bots and Animals,” Science, Vol. 291, 2001, pp. 599-600.
[3]	X. Song, C.-Y. Lin and M.-T. Sun, “Autonom-ous Learning of Visual Concept Models,” Proceedings of IEEE International Symposium on Circuits and Systems, Vol. 5, 2005, pp. 4598-4601.
[4]	B. Bolder, H. Brandl, M. Heracles, H. Janssen, I. Mikhailova, J. Schmudderich and C. Goerick, “Ex-pectation-Driven Autonomous Learning and Interaction Sys-tem,” Proceedings of IEEE-RAS International Conference on Humanoid Robots, Korea, 2008, pp. 553-560.
[5]	P. M. Roth and H. Bischof, “Active Sampling via Tracking,” IEEE-CS Conference on Computer Vision and Pattern Recognition Workshop, Anchorage, 2008, pp. 1-8.
[6]	J.-X. Peng, K. Li and G. W. Irwin, “A Novel Continuous Forward Algorithm for RBF Neural Modeling,” IEEE Transactions on Automatic Control, Vol. 52, No. 1, 2007, pp. 117-122.
[7]	S. Ozawa, S.-L. Toh, S. Abe, S. Pang and N. Kasabov, “Incremental Learning of Feature Space and Classifier for Face Recognition,” Neural Networks, Vol. 18, No. 5-6, 2005, pp. 575-584.
[8]	J. Platt, “A Resource-allocating Network for Function Interpolation,” Neural Computation, Vol. 3, No. 2, 1991, pp. 213-225.
[9]	N. Kasabov, “Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines,” Springer, New York, 2002.
[10]	A. Roy, S. Govil and R. Miranda, “An Algorithm to Generate Radial Basis Function (RBF)-Like Nets for Classification Problems,” Neural Networks, Vol. 8, No. 2, 1995, pp. 179-202.
[11]	S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice-Hall, USA, 1999.
[12]	A. Roy, “Connectionism, Controllers and a Brain Theory,” IEEE Transactions on System, Man and Cybernetics, Part A, Vol. 38, No. 6, 2008, pp. 1434-1441.
[13]	T. Tabuchi, S. Ozawa and A. Roy, “An Autonomous Learning Algorithm of Resource Allocating Net-work,” In: E. Corchado and H. Yin, Eds., Intelligent Data En-gineering and Automated Learning—IDEAL 2009, LNCS, Springer, New York, October 2009, Vol. 5788, pp. 134- 141.
[14]	K. Okamoto, S. Ozawa and S. Abe, “A Fast Incre-mental Learning Algorithm of RBF Networks with Long-Term Memory,” Proceedings of International Joint Conference on Neural Networks, USA, 2003, pp. 102-107.
[15]	A. Asuncion and D. J. Newman, “UCI Machine Learning Repository,” UC, Irvine, School of Information and Computer Science, California, 2007.
[16]	S. Pang, S. Ozawa and N. Kasabov, “Incremental Linear Discriminant Analysis for Classification of Data Streams,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, Vol. 35, No. 5, 2005, pp. 905-914.