Wavelet Chaotic Neural Networks and Their Application to Continuous Function Optimization

Neural networks have been shown to be powerful tools for solving optimization problems. In this paper, we first retrospect Chen's chaotic neural network and then propose several novel chaotic neural networks. Second, we plot the figures of the state bifurcation and the time evolution of most positive Lyapunov exponent. Third, we apply all of them to search global minima of continuous functions, and respectively plot their time evolution figures of most positive Lyapunov exponent and energy function. At last, we make an analysis of the performance of these chaotic neural networks.


INTRODUCTION
Hopfield and Tank first applied the continuous-time, continuous-output Hopfield neural network (HNN) to solve TSP [1], thereby initiating a new approach to optimization problems [2,3].The Hopfield neural network, one of the well-known models of this type, converges to a stable equilibrium point due to its gradient decent dynamics; however, it causes sever local-minimum problems whenever it is applied to optimization problems.M-SCNN has been proved to be more power than Chen's chaotic neural network in solving optimization problems, especially in searching global minima of continuous function and traveling salesman problems [4].
In this paper, we first review the Chen's chaotic neural network.Second, we propose several novel chaotic neural networks.Third, we plot the figures of the state bifurcation and the time evolution of most positive Lyapunov exponent.Fourth, we apply all of them to search global minima of continuous functions, and respectively plot their time evolution figures of most posi-tive Lyapunov exponent and energy function.At last, simulation results are summarized in a Table in order to make an analysis of their performance.

CHAOTIC NEURAL NETWORK MODELS
In this section, several chaotic neural networks are given.And the first is proposed by Chen, the rest proposed by ourselves.
is a steepness parameter of the output function which is varied with different optimization problems.u

Mexican Hat Wavelet Chaotic Neural Network (MHWCNN)
Mexican hat wavelet chaotic neural network is described as follows: ( 1) ( ) ( )( ( ) ) where ( ) are the same with the above.And the Eq.7 is the Shannon wavelet function.

RESEARCH ON CONTINUOUS FUNCTION PROBLEMS
In this section, we apply all the above chaotic neural networks to search global minima of the following three continuous functions.
3) Simulation on the Third Continuous Function The rest parameters are set as follows:  The global minimum and its responding point of the simulation are respectively 0.39789 and (9.4246, 2.4747).The global minimum and its responding point of the simulation are respectively 0.39789 and (3.1413, 2.2733).

ANALYSIS OF THE SIMULATION RESULTS
Simulation results are summarized in Table 1.The columns "GM/ER", "TGM", "PGM" and "AVER" represent, respectively, global minimum/error rate; theoretical global minimum; practical global minimum; average error.Seen from the Table 1, we can conclude that the wavelet chaotic neural networks are superior to Chen's in AVER

CONCLUSION
We have introduced Chen's and wavelet chaotic neural networks.We make an analysis of them in solving continuous function optimization problems, and find out that wavelet chaotic neural networks are superior to Chen's in general. u

1 )I 1 , Figure 2 .I
398 and its responding point are (0, 0), (0.08983, -0.7126) or (-0.08983, 0.7126), (-3.142, 2.275) or (3.142, 2.275) or (9.425, 2.425).In order to make comparison conveniently, we set some parameters such as the annealing speed  , the self-feedback and the initial value of internal state 283, 0.283].Meanwhile, we set the iteration as large as 5000 so as to get stable state of a global minimum.Simulation on the First Continuous Function The rest parameters are set as follows: k =1, =0.5,  =1/10, =0.85.0 The time evolution figures of the biggest positive Lyapunov exponent and energy function of Chen's in solving the first continuous function are shown as Figure The global minimum and its responding point of the simulation are respectively -0.99989 and (0.0073653, 0.0073653).2) Simulation on the Second Continuous Function The rest parameters are set as follows: k =1, =0.02,  =1/20, =0.85.0 The time evolution figures of most positive Lyapunov exponent and energy function of Chen's in

Figure 1 .
Figure 1.Time evolution figure of Lyapunov exponent.

Figure 2 .
Figure 2. Time evolution figure of energy function.

Figure 3 .
Figure 3.Time evolution figure of Lyapunov exponent.

Figure 4 .
Figure 4. Time evolution figure of energy function.solving the first continuous function are shown as Figure 3, Figure 4.The global minimum and its responding point of the simulation are respectively -1 and (0, 0.70712).3)Simulation on the Third Continuous Function The rest parameters are set as follows:

Figure 5 .
Figure 5.Time evolution figure of Lyapunov exponent.

Figure 6 .I
Figure 6.Time evolution figure of energy function.

1 )II
Simulation on the First Continuous Function The rest parameters are set as follows: k =1, =0.5, u =0.5, =0.65.0 The time evolution figures of most positive Lyapunov exponent and energy function of MWCNN in solving the first continuous function are shown as Figure 7, Figure 8.The global minimum and its responding point of the simulation are respectively -0.99997 and (0.0038638, 0.0038638).2)Simulation on the Second Continuous Function The rest parameters are set as follows:k =1, =0.05, u =0.7, =0.2.0The time evolution figures of most positive Lyapunov exponent and energy function of MWCNN in solving the first continuous function are shown as Figure 9, Figure 10.

Figure 7 .
Figure 7. Time evolution figure of Lyapunov exponent.

Figure 8 .
Figure 8.Time evolution figure of energy function.

Figure 9 .
Figure 9.Time evolution figure of Lyapunov exponent.

Figure 10 .
Figure 10.Time evolution figure of energy function.

Figure 11 .
Figure 11.Time evolution figure of Lyapunov exponent.

Figure 12 .I
Figure 12.Time evolution figure of energy function.The global minimum and its responding point of the simulation are respectively -1.0021 and (-0.074007, 0.76863).3)Simulation on the Third Continuous Function The rest parameters are set as follows:

Figure 14 .
Figure 14.Time evolution figure of energy function.

Figure 15 .
Figure 15.Time evolution figure of Lyapunov exponent.

I 0 I
the first continuous function are shown as Figure13, Figure14.The global minimum and its responding point of the simulation are respectively -0.99996 and (0.0043259, 0.0043259).2)Simulation on the Second Continuous Function The rest parameters are set as follows:k =1, =0.05, u =2.8, =0.05.0 The time evolution figures of most positive Lyapunov exponent and energy function of MHWCNN in solving the first continuous function are shown as Figure 15, Figure 16.The global minimum and its responding point of the simulation are respectively -1.0316 and (-0.089825, 0.71263).3) Simulation on the Third Continuous Function.The rest parameters are set as follows: k =1, =0.05, =0.3, =0.2.u The time evolution figures of most positive Lyapunov exponent and energy function of MHWCNN in solving the first continuous function are shown as Figure 17, Figure 18.

Figure 16 .
Figure 16.Time evolution figure of energy function.

Figure 17 .
Figure 17.Time evolution figure of Lyapunov exponent.

Figure 18 .
Figure 18.Time evolution figure of energy function.

Table 1 .
the simulation results of the chaotic neural networks.