Multi-Valued Neuron with Sigmoid Activation Function for Pattern Classification

Shen-Fu Wu, Yu-Shu Chiou, Shie-Jue Lee

Department of Electrical Engineering, National Sun Yat-sen University.

DOI: 10.4236/jcc.2014.24023                

Abstract

Multi-Valued Neuron (MVN) was proposed for pattern classification. It operates with complex-valued inputs, outputs, and weights, and its learning algorithm is based on error-correcting rule. The activation function of MVN is not differentiable. Therefore, we can not apply backpropagation when constructing multilayer structures. In this paper, we propose a new neuron model, MVN-sig, to simulate the mechanism of MVN with differentiable activation function. We expect MVN-sig to achieve higher performance than MVN. We run several classification benchmark datasets to compare the performance of MVN-sig with that of MVN. The experimental results show a good potential to develop a multilayer networks based on MVN-sig.

Keywords

Pattern Classification; Multi-Valued Neuron (MVN); Differentiable Activation Function; Backpropagation

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Aizenberg, N.N. and Aizenberg, I.N. (1992) CNN Based on Multivalued Neuron as a Model of Associative Memory For Grey Scale Images. In Proceedings of Second IEEE International Workshop on Cellular Neural Networks and their Applications (CNNA-92), 36-41. http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=274330
[2]	Aizenberg, I., Aizenberg, N.N. and Vandewalle, J.P. (2000) Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications. Springer. http://dx.doi.org/10.1007/978-1-4757-3115-6
[3]	Aizenberg, I. and Moraga, C. (2007) Multi-layer Feed Forward Neural Network Based on Multi-Valued Neurons (mlmvn) and a Backpropagation Learning Algo-rithm. Soft Computing—A Fusion of Foundations, Methodologies and Applications, 11, 169-183. http://www.springerlink.com/index/T645547TN41006G0.pdf
[4]	Aizenberg, I., Paliy, D.V., Zurada, J.M. and Astola, J. T. (2008) Blur Identification by Multilayer Neural Network Based on Multivalued Neurons. 19, 883-898. http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4479859
[5]	Aizenberg, I. (2010) Periodic Activation Function and a Modified Learning Algorithm for the Multivalued Neuron. 21, 1939-1949. http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=5613940
[6]	Aizenberg, I. (2011) Complex-Valued Neural Networks with Multi-Valued Neurons. Springer.
[7]	Wilson, E. (1994) Backpropagation Learning for Systems with Discrete-Valued Functions. Proceedings of the World Congress on Neural Networks, San Diego, California, June.
[8]	Hagan, M.T., Demuth, H.B., Beale, M.H., et al. (1996) Neural Network Design. Thomson Learning Stamford, CT.
[9]	UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/index.html