Neural Network Approach for Solving Singular Convex Optimization with Bounded Variables

Rendong Ge, Lijun Liu, Yi Xu

School of Science, Dalian Nationalities University, Dalian, China.

DOI: 10.4236/ojapps.2013.33036                

Abstract

Although frequently encountered in many practical applications, singular nonlinear optimization has been always recognized as a difficult problem. In the last decades, classical numerical techniques have been proposed to deal with the singular problem. However, the issue of numerical instability and high computational complexity has not found a satisfactory solution so far. In this paper, we consider the singular optimization problem with bounded variables constraint rather than the common unconstraint model. A novel neural network model was proposed for solving the problem of singular convex optimization with bounded variables. Under the assumption of rank one defect, the original difficult problem is transformed into nonsingular constrained optimization problem by enforcing a tensor term. By using the augmented Lagrangian method and the projection technique, it is proven that the proposed continuous model is convergent to the solution of the singular optimization problem. Numerical simulation further confirmed the effectiveness of the proposed neural network approach.

Keywords

Neural Networks;Singular Nonlinear Optimization;Stationary Point;Augmented Lagrangian Function;Convergence; LaSalle’s Invariance Principle Plain

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	R. B. Schnabel and T.-T. Chow, “Tensor Methods for Unconstrained Optimization Using Second Derivatives,” SIAM: SIAM Journal on Optimization, Vol. 1, No. 3, 1991, pp. 293-315. doi:10.1137/0801020
[2]	D. Feng and R. B. Schnabel, “Tensor Methods for Equa lity Constrained Optimization,” SIAM: SIAM Journal on Optimization, Vol. 6, No. 3, 1996, pp. 653-673. doi:10.1137/S1052623494270790
[3]	A. Bouaricha, “Tensor Methods for Large Sparse Uncon strained Optimization,” SIAM: SIAM Journal on Optimi zation, Vol. 7, No. 3, 1997, pp. 732-756. doi:10.1137/S1052623494267723
[4]	R. Ge and Z. Xia, “Solving a Type of Modified BFGS Algorithm with Any Rank Defects and the Local Q-Su perliner Convergence Properties,” Journal of Computa tional and Applied Mathematics, Vol. 22, No. 1-2, 2006, pp. 193-208.
[5]	R. Ge and Z. Xia, “A Type of Modified BFGS Algorithm with Rank Defects and Its Global Convergence in Convex Minimization,” Journal of Pure and Applied Mathematics: Advances and Applications, Vol. 3, No. 1, 2010, pp. 17-35.
[6]	Y. S. Xia and J. Wang, “On the Stability of Globally Pro jected Dynamical Systems,” Journal of Optimization Theory and Applications, Vol. 106, No. 1, 2000, pp. 129-150. doi:10.1023/A:1004611224835
[7]	Y. S. Xia and J. Wang, “A New Neural Network for Solv ing Linear Programming Problems and Its Applications,” IEEE Transactions on Neural Networks, Vol. 7, No. 2, 1996, pp. 525-529. doi:10.1109/72.485686
[8]	Y. S. Xia, “A New Neural Network for Solving Linear and Quadratic Programming Problems,” IEEE Transac tions on Neural Networks, Vol. 7, No. 6, 1996, pp. 1544-1547. doi:10.1109/72.548188
[9]	Y. S. Xia, J. Wang, “A General Methodology for De signing Globally Convergent Optimization Neural Net works,” IEEE Transactions on Neural Networks, Vol. 9, No. 6, 1998, pp. 1331-1343. doi:10.1109/72.728383
[10]	Y. S. Xia, “A Recurrent Neural Network for Solving Lin ear Projection Equations,” IEEE Transactions on Neural Networks Vol. 13, No. 3, 2000, pp. 337-350. doi:10.1016/S0893-6080(00)00019-8
[11]	Y. S. Xia, H. Leung and J. Wang, “A Projection Neural Network and Its Application to Constrained Optimization Problems,” IEEE Transactions on Circuits and Systems I, Vol. 49, No. 4, 2002, pp. 447-458. doi:10.1109/81.995659
[12]	Y. S. Xia and J. Wang, “A General Projection Neural Network for Solving Monotone Variational Inequality and Related Optimization Problems,” IEEE Transactions on Neural Networks, Vol. 15, No. 2, 2004, pp. 318-328. doi:10.1109/TNN.2004.824252
[13]	X. Gao, L. Z. Liao and W. Xue, “A Neural Network for a Class of Convex Quadratic Minimax Problems with Con straints,” IEEE Transactions on Neural Networks, Vol. 15, No. 3, 2004, pp. 622-628. doi:10.1109/TNN.2004.824405
[14]	C. Y. Sun and C. B. Feng, “Neural Networks for Non convex Nonlinear Programming Problems: A Switching Control Approach,” Lecture Notes in Computer Science, Vol. 3496, 2005, pp. 694-699. doi:10.1007/11427391_111
[15]	Q. Tao, X. Liu and M. S. Xue, “A Dynamic Genetic Al gorithm Based on Continuous Neural Networks for a Kind of Non-Convex Optimization Problems,” Applied Mathematics and Computation, Vol. 150, No. 3, 2004, pp. 811-820. doi:10.1016/S0096-3003(03)00309-6
[16]	D. J. Bell and D. H. Jacobson, “Singular Optimal Control Problems,” Academic Press, New York, 1975.
[17]	F. Lamnabhi-Lagarrigue and G. Stefani, “Singular Optimal Control Problem: On the Necessary Conditions of Optimality,” SIAM: SIAM Journal on Control and Opti mization, Vol. 28, No. 4, 1990, pp. 823-840. doi:10.1137/0328047
[18]	L. Liu, R. Ge and P. Gao, “A Novel Neural Network for Solving Singular Nonlinear Convex Optimization Prob lems,” Lecture Notes in Computer Science, Vol. 7063, 2011, pp. 554-561. doi:10.1007/978-3-642-24958-7_64
[19]	D. Kinderlehrer and G. Stampacchia, “An Introduction to Variational Inequalities and Their Applications,” Aca demic Press, New York, 1980.
[20]	X. Du, Y. Yang and M. Li, “Further Studies on the Heste nes-Powell augmented Lagrangian Function for Equality Constraints in Nonlinear Programming Problems,” OR Transactions, Vol. 10, No. 1, 2006, pp. 38-46.
[21]	J. J. More, B. S. Garbow and K. E. Hillstrom, “Testing Unconstrained Optimization Software,” ACM Transac tions on Mathematical Software, Vol. 7, No. 1, 1981, pp. 19-31. doi:10.1145/355934.355936