Share This Article:

LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy

Abstract Full-Text HTML XML Download Download as PDF (Size:460KB) PP. 19-28
DOI: 10.4236/jilsa.2013.51003    4,637 Downloads   6,363 Views   Citations


In this paper we study the problem of model selection for a linear programming-based support vector machine for regression. We propose generalized method that is based on a quasi-Newton method that uses a globalization strategy and an inexact computation of first order information. We explore the case of two-class, multi-class, and regression problems. Simulation results among standard datasets suggest that the algorithm achieves insignificant variability when measuring residual statistical properties.

Conflicts of Interest

The authors declare no conflicts of interest.

Cite this paper

P. Rivas-Perea, J. Cota-Ruiz, J. Venzor, D. Chaparro and J. Rosiles, "LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy," Journal of Intelligent Learning Systems and Applications, Vol. 5 No. 1, 2013, pp. 19-28. doi: 10.4236/jilsa.2013.51003.


[1] A. J. Smola and B. Scholkopf, “A Tutorial on Support Vector Regression,” Statistics and Computing, Vol. 14, No. 3, 2004, pp. 199-222. doi:10.1023/B:STCO.0000035301.49549.88
[2] D. Anguita, A. Boni, S. Ridella, F. Rivieccio and D. Sterpi, “Theoretical and Practical Model Selection Methods for Support Vector Classifiers,” Support Vector Machines: Theory and Applications, Vol. 177, 2005, pp. 159-179. doi:10.1007/10984697_7
[3] K. Duan, S. Keerthi and A. Poo, “Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters,” Neurocomputing, Vol. 51, 2003, pp. 41-59. doi:10.1016/S0925-2312(02)00601-X
[4] Z. Hui-ren and P. Zheng, “Method for Selecting Parameters of Least Squares Support Vector Machines Based on GA and Bootstrap,” Journal of System Simulation, Vol. 12, 2008.
[5] D. Anguita, S. Ridella, F. Rivieccio and R. Zunino, “Hyperparameter Design Criteria for Support Vector Classifiers,” Neurocomputing, Vol. 55, No. 1-2, 2003, pp. 109-134. doi:10.1016/S0925-2312(03)00430-2
[6] L. Wang and S. O. Service, “Support Vector Machines: Theory and Applications,” Studies in Fuzziness and Soft Computing, Springer-Verlag, Berlin, 2005.
[7] G. Cawley, “Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted Ls-Svms,” IEEE International Conference on Neural Networks, 16-21 July 2006. doi:10.1109/IJCNN.2006.246634
[8] P. R. Perea, “Algorithms for Training Large-Scale Linear Programming Support Vector Regression and Classification,” Ph.D. Thesis, The University of Texas, El Paso, 2011.
[9] J. Dennis and R. Schnabel, “Numerical Methods for Unconstrained Optimization and Nonlinear Equations,” Society for Industrial Mathematics, 1996. doi:10.1137/1.9781611971200
[10] M. Argaez and L. Velazquez, “A New Infeasible InteriorPoint Algorithm for Linear Programming,” Proceedings of the 2003 Conference on Diversity in Computing, ACM, New York, 2003, pp. 12-14.
[11] J. Mercer, “Functions of Positive and Negative Type, and Their Connection with the Theory of Integral Equations,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, Vol. 209, No. 441-458, 1909, pp. 415-446. doi:10.1098/rsta.1909.0016
[12] R. Courant and D. Hilbert, “Methods of Mathematical Physics,” Interscience, New York, 1966.
[13] Z. Lu, J. Sun and K. R. Butts, “Linear Programming Support Vector Regression with Wavelet Kernel: A New Approach to Nonlinear Dynamical Systems Identification,” Mathematics and Computers in Simulation, Vol. 79, No. 7, 2009, pp. 2051-2063. doi:10.1016/j.matcom.2008.10.011
[14] Y. Torii and S. Abe, “Decomposition Techniques for Training Linear Programming Support Vector Machines,” Neurocomputing, Vol. 72, No. 4-6, 2009, pp. 973-984. doi:10.1016/j.neucom.2008.04.008
[15] L. Zhang and W. Zhou, “On the Sparseness of 1-Norm Support Vector Machines,” Neural Networks, Vol. 23, No. 3, 2010, pp. 373-385.
[16] T. Fawcett, “Roc Graphs: Notes and Practical Considerations for Researchers,” Machine Learning, Vol. 31, 2004, pp. 1-38.
[17] J. Nocedal and S. Wright, “Numerical Optimization,” Springer Verlag, New York, 1999. doi:10.1007/b98874
[18] M. Hestenes, “Pseudoinversus and Conjugate Gradients,” Communications of the ACM, Vol. 18, No. 1, 1975, pp. 40-43. doi:10.1145/360569.360658

comments powered by Disqus

Copyright © 2018 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.