FPGA Simulation of Linear and Nonlinear Support Vector Machine

Davood Mahmoodi, Ali Soleimani, Hossein Khosravi, Mehdi Taghizadeh

.

DOI: 10.4236/jsea.2011.45036                

Abstract

Simple hardware architecture for implementation of pairwise Support Vector Machine (SVM) classifiers on FPGA is presented. Training phase of the SVM is performed offline, and the extracted parameters used to implement testing phase of the SVM on the hardware. In the architecture, vector multiplication operation and classification of pairwise classifiers is designed in parallel and simultaneously. In order to realization, a dataset of Persian handwritten digits in three different classes is used for training and testing of SVM. Graphically simulator, System Generator, has been used to simulate the desired hardware design. Implementation of linear and nonlinear SVM classifier using simple blocks and functions, no limitation in the number of samples, generalized to multiple simultaneous pairwise classifiers, no complexity in hardware design, and simplicity of blocks and functions used in the design are view of the obvious characteristics of this research. According to simulation results, maximum frequency of 202.840 MHz in linear classification, and classification accuracy of 98.67% in nonlinear one has been achieved, which shows outstanding performance of the hardware designed architecture.

Keywords

Hardware Architecture, Support Vector Machine, FPGA, Persian Handwritten Digits, System Generator

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	M. Ruiz-Llata and M. Yébenes-Calvino, “FPGA Implementation of Support Vector Machines for 3D Object Identification,” Artificial Neural Networks–ICANN 2009, Limassol, 14-17 September, 2009, pp. 467-474.
[2]	U. Meyer-Baese, “Digital Signal Processing with Field Programmable Gate Arrays,” Springer Verlag, Berlin, 2007.
[3]	A. Ganapathiraju, et al., “Applications of Support Vector Machines to Speech Recognition,” IEEE Transactions on Signal Processing, Vol. 52, No. 8, 2004, pp. 2348-2355. doi:10.1109/TSP.2004.831018
[4]	O. Pina-Ramfrez, et al., “An FPGA Implementation of Linear Kernel Support Vector Machines,” IEEE International Conference on the Reconfigurable Computing and FPGA’s, San Luis, September 2006, pp. 1-6.
[5]	D. Anguita, et al., “A Digital Architecture for Support Vector Machines: Theory, Algorithm, and FPGA Implementation,” Neural Networks, IEEE Transactions on, Vol. 14, No. 5, 2003, pp. 993-1009. doi:10.1109/TNN.2003.816033
[6]	F. M. Khan, et al., “Hardware-Based Support Vector Machine Classification in Logarithmic Number Systems,” Circuits and Systems, Vol. 5, 2004, pp. 5154-5157.
[7]	C. F. Hsu, et al., “Support Vector Machine FPGA Implementation for Video Shot Boundary Detection Application,” System-on-Chip Conference, 4-5 November 2009, pp. 239-242.
[8]	M. Moradi, et al., “A New Method of FPGA Implementation of Farsi Handwritten Digit Recognition,” European Journal of Scientific Research, Vol. 39, No. 3, 2010, pp. 309-315.
[9]	V. Vapnik, “Statistical Learning Theory,” John Wiley & Sons Inc., New York, 1998.
[10]	V. Vapnik, “The Nature of Statistical Learning Theory,” Springer Verlag, Berlin, 1995.
[11]	S. Abe, “Support Vector Machines for Pattern Classification (Advances in Pattern Recognition),” Springer-Verlag New York, 2005.
[12]	M. Martínez-Ramón and C. G. Christodoulou, “Support Vector Machines for Antenna Array Processing and Electromagnetics,” Morgan & Claypool, California, 2006.
[13]	J. Diederich, “Rule Extraction from Support Vector Machines,” Springer-Verlag, New York, 2008. doi:10.1007/978-3-540-75390-2
[14]	B. Sch?lkopf and A. J. Smola, “Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond,” The MIT Press, Cambridge, 2002.
[15]	T. M. Huang, et al., “Kernel Based Algorithms for Mining Huge Data Sets: Supervised, Semi-Supervised, and Unsupervised Learning,” Springer Verlag, Berlin, 2006.
[16]	U. H. G. Kre?el, “Pairwise Classification and Support Vector Machines,” MIT Press, Cambridge, 1999.
[17]	C. Chih-Chung and L. Chih-Jen, “LIBSVM: A Library for Support Vector Machines,” 2001. http://www.csie.ntu.edu.tw/~cjlin/libsvm
[18]	H. Khosravi and E. Kabir, “Introducing a Very Large Dataset of Handwritten Farsi Digits and a Study on Their Varieties,” Pattern Recognition Letters, Vol. 28, 2007, pp. 1133-1141. doi:10.1016/j.patrec.2006.12.022
[19]	S. Theodoridis and K. Koutroumbas, “Pattern Recognition,” 3rd Edition, Elsevier, Amsterdam, 2006.
[20]	H. T. Lin and C. J. Lin, “A Study on Sigmoid Kernels for SVM and the Training of Non-PSD Kernels by SMO- Type Methods,” Department of Computer Science and Information Engineering, National Taiwan University, Taiwan, 2003.