Bifurcation Analysis of Reduced Network Model of Coupled Gaussian Maps for Associative Memory

This paper proposes an associative memory model based on a coupled system of Gaussian maps. A one-dimensional Gaussian map describes a discrete-time dynamical system, and the coupled system of Gaussian maps can generate various phenomena including asymmetric fixed and periodic points. The Gaussian associative memory can effectively recall one of the stored patterns, which were triggered by an input pattern by associating the asymmetric two-periodic points observed in the coupled system with the binary values of output patterns. To investigate the Gaussian associative memory model, we formed its reduced model and analyzed the bifurcation structure. Pseudo-patterns were observed for the proposed model along with other conventional associative memory models, and the obtained patterns were related to the high-order or quasi-periodic points and the chaotic trajectories. In this paper, the structure of the Gaussian associative memory and its reduced models are introduced as well as the results of the bifurcation analysis are presented. Furthermore, the output sequences obtained from simulation of the recalling process are presented. We discuss the mechanism and the characteristics of the Gaussian associative memory based on the results of the analysis and the simulations conducted.


Previous studies have proposed multiple-associative memory models based on
https://doi.org/10.4236/ijmnta.2019.81001Int.J. Modern Nonlinear Theory and Application dynamical systems [1] [2] [3] [4].J. J. Hopfield proposed a neural network model which is popularly referred to as an associative memory [5].The Hopfield model functions as an associative memory that mimics the human brain.Hebbian learning can be used to calculate the weight of connections between neurons using stored patterns.In the recalling process of the Hopfield model, the system needs to be iteratively updated until its energy function, which is guaranteed to decrease as updating continues, converges to a local minimum.Because the local minimum corresponds to a stored pattern, the Hopfield associative memory can recall one of the stored patterns depending on the initial values of the network.This is called auto-association.
Hopfield model has been extended to develop bidirectional and multidirectional associative memories [6] [7] [8] [9].The bidirectional associative memory comprises a two-layer network and the model can recall another pattern by using input pattern.This is referred to as hetero-association.The size of the recalling pattern can be different from that of the stored pattern.On the other hand, the multidirectional associative memory has an extended structure of the bidirectional model with three or more layers.The associative structure of multidirectional model is more complicated than the bidirectional model, and the model can successively recall another pattern as it is being iteratively updated.In contrast to the conventional associative models, a chaotic associative memory model, which is made up of chaotic neurons, has been proposed as a non-periodic associative memory [10].While the conventional models adopted the Hebbian learning in calculating the weight between neurons, it incorporates an auto-association matrix into the chaotic neuron associative memory model.One of the significant characteristics of chaotic associative memory is that it can successively recall stored patterns with transforming the output patterns depending on the system parameters and the chaotic neuron dynamics.Another characteristic of the chaos associative memory is that it can recall not only stored patterns but also reverse patterns of the stored patterns.
The retrieval ability and the storage capacity are important points to be considered when evaluating associative memory models and have been the subject of many studies.One major problem associated with the associative memory model is the generation of the pseudo-patterns.How to avoid the generation of the pseudo-patterns has received remarkable attention in associative memory research community.
However, the mechanisms that are used to effectively recall the stored patterns and to generate pseudo-patterns have not been well elucidated.Herein, we propose a novel associative memory model comprised of Gaussian map to investigate the mechanism of the associative memory based on the qualitative bifurcation theory.
The Gaussian map is a one-dimensional dynamical system which generates various phenomena including periodic points and chaos [11] [12].Asymmetric fixed and periodic points are observed under the particular parameter settings when investigating the dynamics of the coupled Gaussian maps.By associating the asymmetric values of the coupled Gaussian maps with the binary values of the recalling patterns, the Gaussian associative memory can successfully be applied in the recalling process as well as the other conventional methods.The Gaussian associative memory has a simple structure as the Gaussian map is a one-dimensional dynamical system and shows very complicated phenomena such as chaotic behavior and quasi-periodic oscillations.Furthermore, the bifurcation structure of the associative memory can be qualitatively analyzed by considering the reduced model of the Gaussian associative memory.
This study investigates the bifurcation structure of periodic points observed in the high dimensional coupled network of the Gaussian maps that can be used to recall stored patterns.Although multiple associative memory models have been proposed, the detailed bifurcation analysis of these models has not been conducted; the analyses conducted on models are often based on empirical approaches.As a result, we focus on the bifurcation analysis of the proposed high dimensional coupled network to propose and verify the reduced model of the Gaussian associative memory model.In the proposed model, successful retrieval of the stored patterns is associated with the existence of stable asymmetric two-periodic points, which are observed in a coupled system of Gaussian maps.
In addition, when the Gaussian associative memory recalls the pseudo-patterns, they correspond to high order periodic points and chaotic behavior in the coupled maps.
First, this study discusses the structure of the Gaussian associative memory and further addresses the manner in which its reduced model can be formed.
Subsequently, the results of the bifurcation analysis of the reduced model are presented.Finally, we demonstrate the recalling process of the Gaussian associative memory while considering the noisy patterns as the input pattern.

Model Description
In this section, we introduce the dynamics of the coupled Gaussian maps for an associative memory and its reduced model.

Coupled Gaussian Maps for Associative Memory
We proposed a Gaussian associative memory model composed of Gaussian maps [13].The dynamics of ith Gaussian map in the Gaussian associative memory can be described as follows: where α , β , and ε denote parameters; N is the number of Gaussian maps in the network; p is the number of the stored patterns; ij w is the following symmetric auto-associative matrix [10]: where k i ξ represents the ith pixel of the kth stored pattern with a discrete value of zero or one.We adopted the stored binary patterns as shown in Figures Gaussian maps so that each pixel in the patterns corresponds to each Gaussian map.For the recalling process, when the value of i x is higher (lower) than the threshold, the ith pixel is white (black) in the output pattern.We set the threshold to −0.5 in the experiments conducted.

Reduced Model of Gaussian Associative Memory
We     1(a)-(d).
The dynamics of the reduced model of the Gaussian associative memory can be described as a difference equation, which can be expressed as follows: ( ) ( ) ( ) Equivalently, they can be described as an iterated map, which can be expressed as follows: ( ) where t denotes discrete time,  represents a set of real numbers, and x and f represent ( ) , , , x x x Τ  and ( ) , respectively.The dynamics of the reduced system of Gaussian associative memory discussed herein are described as where i h is the correction coefficient which is equal to the number of Gaussian maps in each group as shown in the fourth column of Table 1.

Method
This section presents the method of bifurcation analysis for analyzing the reduced model of the Gaussian associative memory and the necessary conditions and the parameter settings used in the simulation analysis of the Gaussian Associative memory.A notation list for all indices and parameters used in this paper is shown in Table 2.

Bifurcation Analysis
In bifurcation analysis, we used a method based on the qualitative bifurcation theory [14].The point * x satisfying ( ) becomes a fixed point in Equation (5).The characteristic equation for the fixed point * x is defined as where I is the 16 × 16 identity matrix and Df denotes the derivative of f .We consider * x to be hyperbolic if none of the absolute eigenvalues of Df are at unity.Note that in Equation ( 6), an m-periodic point can be investigated by replacing f with m f , i.e., the mth iteration of f .In the following discussion, we only consider the properties of a fixed point of f , though a similar argument can be applied to a periodic point of f .
Let us consider the topological classification of a hyperbolic fixed point * x .
The topological type of a hyperbolic fixed point is determined by dim u E and det u L , where u E is the intersection of 16  and the direct sum of the generalized eigenspaces of ( )

Df x
corresponding to eigenvalue m such that ( )  When we consider the distribution of the characteristic multipliers of Equation (7), D and I correspond to the even and odd numbers, respectively, of the characteristic multipliers on the real axis ( ) and k represents the number of the characteristic multipliers outside the unit circle on the complex plane.When all characteristic multipliers are in the unit circle, the topological type is 0 D that means completely stable; otherwise, , 0 k D k > and , 0 k I k > represent directly unstable and inversely unstable, respectively.
Bifurcation occurs when the topological type of a fixed point is changed by the varying of a system parameter.In the reduced model, co-dimension-one bifurcations, i.e., tangent and period-doubling bifurcations, are observed when hyperbolicity of the system is destroyed.This corresponds to the critical distribution of the characteristic multiplier µ such that 1 µ = + for tangent bifurcation and 1 µ = − for period-doubling bifurcation.
The bifurcation sets of a fixed point were computed by solving the simulta-neous Equations ( 6) and (7).For numerical determination [14], we used Newton-Raphson method.The Jacobian matrix of the set of equations was derived from the first and second derivatives of map f.

Simulation Settings
With respect to the Gaussian associative memory defined by Equations ( 1) and (2), we evaluated the recalling ability of the Gaussian associated memory for the 10 × 10 stored patterns as shown in Figure 1.The parameters were set as follows: .The white and black pixels of an initial input pattern were converted to ( ) and −1, respectively.As an initial input pattern, we used the noisy pattern which includes inverted bits compared with the original stored pattern as shown in Figures 1(a)-(d).When the noisy pattern includes no inverted bits, the pattern is completely the same as the stored pattern.When we increase the ratio of the noise added in the initial pattern, the noisy pattern finally becomes the completely inverted pattern of the original stored pattern.
We investigated the relationship between the number of the inverted bits included in the noisy pattern and recalling probability in the simulation conducted.With each noisy pattern containing 0 to 100 inverted bits, the Gaussian associative memory model was run for 100 iterations.When the output pattern set at 100 t = was exactly same to the stored pattern, we added one to a variable S.After executing for 1000 times executions, the average value of the recalling probability was calculated as S/1000.In Figure 4(a), each value of i x is slightly different from one other.That is because the coefficients represented by ij w and i h represent the pixel values of the stored patterns while the number of Gaussian maps is represented by one reduced map, respectively so that the reduced model does not have complete symmetricity.Therefore, i x described in Equation ( 5) becomes different from each other.However, to distinguish the fixed points shown in Figure 4(a) and    G represented by as shown in Figure 5, and the shaded regions in Figure 6 and Figure 7.The asymmetric fixed point that appears in Figure 6 is the only the fixed point that represents exactly the same pattern represented by one of the stored patterns.The rest are the pseudo-patterns.Note that the asymmetric fixed points corresponding to other stored patterns also exist in approximately the same parameter regions, however, they were omitted here.

Bifurcation Structure of Fixed Points
In Figure 6, in the right-hand side of the parameter region where the stable asymmetric fixed point exits, multiple period doubling bifurcations appeared, which were represented by 1 1 I to 1 16 I in a clockwise direction.By going through the period doubling bifurcations with a changing parameter, the asymmetric fixed point became unstable.After passing through eight period doubling bifurcations from 1 1 I to 1 8 I , the asymmetric fixed point lost its stability and became 8 D .The ends of the 16 curves of period doubling bifurcations con- verged at ( ) ( ) . The reduced model loses its coupled structure when 0 ε = ., the two-periodic points were in-phase while the phases were shifted at 0.75 ε = and 0.8 ε = .Figure 10 shows the bifurcation diagrams of two-periodic points observed in the reduced model as each stored pattern in Figures 1(e)-(h) was used as an initial pattern.In the Gaussian associative memory model, the oscillatory responses  of the asymmetric two-periodic points can be separated by setting a threshold so that the Gaussian memory model can generate output patterns identical to the stored patterns.In the shaded regions in the bifurcation diagrams shown in Figure 10, the regions where stable asymmetric two-periodic points exist can be seen as partly over-wrapped.The shapes of the regions where stable asymmetric two-periodic points appear are similar to the downward triangles, and the regions are surrounded by period doubling and tangent bifurcations at the left and right-hand sides, respectively.

Bifurcation Structure of Two-Periodic Points
The existence of the stable asymmetric two-periodic points is the key for  making the Gaussian associative model to work effectively.At the same time, the coupled system also has the stable symmetric and asymmetric fixed points.Nevertheless, the stable symmetric fixed point is not appropriate for the associative memory because the output values of each Gaussian map cannot be separated into binary values.In contrast, the asymmetric fixed point seems to be appropriate for the associative memory.However, as shown in Figure 6  I as shown in Figure 6.Based on the results obtained, we found that the other stable asymmetric two-periodic points can also be caused by a stable asymmetric fixed point.

Simulation
We demonstrate the retrieval process of the Gaussian associative memory by using the Gaussian associative memory for 10 × 10 stored patterns as shown in Figure 1.In Figures 11(a)-(d), the initial patterns at 0 t = include 30 pixels of the inverted bits compared with the stored patterns.In each case, the Gaussian associative memory could recall the corresponding stored pattern at 2 t = .In contrast, when the initial pattern includes higher rate inverted bits, they converged to the reverse pattern of the corresponding stored pattern.When the rate of the noise in the initial pattern is around 50%, pseudo-patterns rarely appeared.

Conclusions
A novel associative memory model based on the Gaussian coupled maps was proposed.We explored the characteristics of the Gaussian associative memory by investigating its behavior as the number of coupled maps is reduced.When 10 × 10 pixel patterns were stored into the associative memory, each map corresponding to the input pixel was classified into 16 groups based on their synchronization, therefore, we can consider the model as a 16-coupled Gaussian map.The Gaussian associate memory can be simplified by reducing the number of coupled maps which makes it possible to investigate its bifurcation structure by analyzing the reduced model.Based on the results of the analysis, we found the parameter region where the stable asymmetric two-periodic points occurred.The stable asymmetric two-periodic points were appropriate in generating the output patterns corresponding to the binary stored patterns because the trajectories can effectively be separated by setting a threshold.In addition, because the symmetric fixed point did not appear in the parameter region, it is preferable for recalling the stored patterns.We demonstrated the retrieval process of the proposed model by conducting a simulation which showed that the Gaussian associative memory could retrieve stored patterns from the noisy pattern including 30% different pixels.
Hence, it can be concluded that the Gaussian associative memory can efficiently recall stored patterns.However, pseudo-patterns were generated with input patterns having high noise rates.The trajectories associated with those pseudo-patterns were high order periodic points or chaotic behavior.Our future work would focus on investigating methods for preventing the generation of those pseudo-patterns, and to how to enlarge the capacity of the memory.For the former task, the method for avoiding chaos or high order periodic points must work well in preventing the generation of pseudo-patterns too.For the latter, the basin of the attractors and more precise bifurcation analysis would be investigated.
How to cite this paper: Kobayashi, M. and Yoshinaga, T. (2019) Bifurcation Analysis of Reduced Network Model of Coupled Gaussian Maps for Associative Memory.International Journal of Modern Nonlinear Theory and Application, 8, 1-16.

1
(a)-(d).Figures 1(e)-(g) are the reduced patterns of Figures 1(a)-(d), which would be addressed later.To remember the patterns with 10 × 10 pixels as shown in Figures 1(a)-(d), the Gaussian associative memory is composed of 100

Figure 2 .
Figure 2. Number of each pixel of (a) stored patterns and (b) reduced patterns.

Figure 3 .
Figure 3. Classification of Gaussian maps corresponding to 10 × 10 pixels when the patterns in Figure 1 were stored in the Gaussian associative memory.White pixels in each panel correspond to the Gaussian maps which have the same input binary values for stored patterns.
stored patterns α , β , and ε system parameters of the Gaussian associative memory k i ξ ith pixel of kth input binary pattern with a discrete value of zero or one unstable fixed point P 1 , P 2 , P 3 , and P 4 stored patterns with 10 × 10 pixels P r1 , P r2 , P r3 , and P r4 reduced patterns of stored patterns with 4 × 4 pixels m l G tangent bifurcation of m-periodic point m l I period-doubling bifurcation of m-periodic point

Figure 4
Figure 4 shows typical fixed points observed in the reduced model.The fixed points can be obtained by setting initial values arbitrarily to

Figure 4 (
Figure 4(b), let us call them symmetric and asymmetric fixed points, respectively.We investigated the bifurcation sets of the reduced network of Gaussian maps for associative memory on the ( ) , β ε -plane using bifurcation analysis as ex- plained in Section 3.1.In the bifurcation diagrams, we use the following symbols: m l G tangent bifurcation of the m-periodic point.m l I period-doubling bifurcation of the m-periodic point.where l distinguishes the same types of the bifurcation sets of the m-periodic points.

Figure 4 .
Figure 4. Fixed points observed in the reduced model.(a) Shows the symmetric fixed pint that appears in Figure 5; and (b) Corresponds to the asymmetric fixed point as shown in Figure 6.

Figures 5 -
Figures 5-7 represent the bifurcation structures of the fixed points on the ( ) , β ε -plane, which were observed in the reduced model.In Figure 5, the stable symmetric fixed point was observed in the region represented by .In contrast, the asymmetric fixed points were observed in the region between 1 1 I and Figures 8(d)-(f) correspond to the 10 × 10 pixel patterns of Figures 8(a)-(c).

Figure 9
Figure 9 shows the different types of waveforms of stable asymmetric two-periodic points when the pattern of Figure 1(e) was used as an initial pattern at 0.4 β = − and 16 α = .The waveforms of Figures 9(a)-(c) were ob-

Figure 5 .
Figure 5. Bifurcation structure of the symmetric and asymmetric fixed points.The asymmetric fixed point is related to the output pattern as shown in Figure 8(a).

Figure 6 .
Figure 6.Bifurcation structure near the parameter regions of stable asymmetric fixed point corresponding to the stored pattern as shown in Figure 8(b).

Figure 7 .
Figure 7. Bifurcation structure corresponding to the asymmetric fixed point associated with the pseudo pattern in Figure 8(c).

1 I and 2 1 G 1 16I 4 I and 2 4 G
and Figure7, the parameter region where the stable asymmetric fixed point appears over-wraps the region where the stable symmetric fixed point exists.We cannot choose the parameter values in such parameter regions because the trajectories might be attracted to the symmetric fixed points.Hence, we decided to set the parameter recalling process of the Gaussian associative memory.To investigate the generation mechanism of the asymmetric two-periodic points, we focused on the results obtained when the pattern shown in Figure1(f) was used as an initial pattern.In Figure10(b), the stable asymmetric two-periodic point that exists in the region surrounded by 2 was derived from the period doubling bifurcation of the stable asymmetric fixed point, represented by as shown in Figure6.On the other hand, the stable asymmetric two-periodic points between 2 in Figure10(b) was generated from 1 1

Figure 9 .
Figure 9.Typical waveforms observed in the reduced model of the Gaussian associative memory as the initial pattern P 1 was set at 0.4 β = − .(a) in-phase stable two-periodic points, (b) and (c) out-of-phase stable two-periodic points.

Figure 10 .
Figure 10.Bifurcation diagram of two-periodic point corresponding to initial patterns of Figures 1(e)-(h).

Figure 11 .
Figure 11.Output sequence obtained using the recalling process of the Gaussian associative memory with 10 × 10 stored patterns.(a)-(d) When 30 pixels of the initial patterns were randomly inverted from the corresponding stored pattern; (e) When 50 pixels are inverted from the stored pattern, P 1 .

Figure 13 (
Figure13(a) and Figure13(b) represent the recalling probability of noisy patterns made from a stored pattern P 1 as shown in Figure1(a) on the Gaussian and Hopfield associative memory models, respectively.In Figure13(a) and Figure13(b), the retrieval ability was 100% up to 20 inverted bits, however, the retrieval ability gradually declined as the number of inverted bits increased.The curve with the title "Others" in the diagram represents the recalling probability of the pseudo-patterns.The curves for the recalling probability of pseudo-patterns have two peaks at around 40 and 60 inverted bits.Comparing the Gaussian and Hopfield associative memory, the recalling probability of pseudo-patterns in the Gaussian associative memory is lower than that of Hopfield associative memory.In contrast, the recalling probability of other stored patterns of the Gaussian associative memory is higher than that of Hopfield associative memory at around 50 inverted bits.In terms of pseudo-pattern generation, the Gaussian associative memory outperformed Hopfield associative memory.

Figure 12 .
Figure 12.Waveform corresponding to the recalling process as shown in Figure 11(e).The trajectories are chaotic and pseudo-pattern output.

Figure 13 .
Figure 13.Relationship between the number of inverted bits and recalling probability.(a) Gaussian associative memory and (b) Hopfield associative memory.

Table 1 .
The relationship between the pixel number in 10 × 10 coupled network and the number of maps in 4 × 4 reduced model.
multiple Gaussian maps in the group could be represented by a representative Gaussian map.As a result, the reduced stored patterns are obtained.Figures1(e)-(h)show the reduced patterns with 4 × 4 pixels, corresponding to the stored patterns as shown in Figures )

Table 2 .
the hyperbolic fixed point is called D-type and I-type, respectively.Based on this definition, we have 33 topologically different Notation list for indices and parameters.