Neural-Based Routing Method for Alleviating Congestion in Complex Networks

In this paper, we propose a new packet routing strategy that incorporates memory information for reducing congestion in communication networks. First, we study the conventional routing strategy which selects the paths for transmitting packets to destinations using the distance information and the dynamical information such as the number of accumulating packets at adjacent nodes. Then, we evaluate the effectiveness of this routing strategy for the scale-free networks. From results of numerical simulations, we conclude that this routing strategy is not effective when the density of the packets increases due to the impermeability of the communication network. To avoid this undesirable problem, we incorporate memory information to the routing strategy. By using memory information effectively, packets are spread into the communication networks, achieving a higher performance than conventional routing strategies for various network topologies, such as scale-free networks, small-world networks, and scale-free networks with community structures.


Introduction
The transmission rate of data packets in communication networks is drastically increasing due to the widespread use of mobile devices.If the number of packets increases sufficiently, then packet congestion occurs in communication networks.Transmissions of packets to their destinations are then blocked, and in the worst case, the packets may be removed from the network by a buffer constraint or a time-to-live constraint.Therefore, the study and analysis of packet congestion has been given significant attention in recent years [1]- [17].
The shortest path algorithms adopted in real-world systems, such as the Dijkstra [18], Bellman-Ford [19], and Warshall-Floyd [20] algorithms, face serious challenges if the number of data packets continues to increase [15].Specifically, these shortest path algorithms transmit packets using only information regarding distances in the communication network, and so routers that lie on a large number of shortest paths easily become congested.Thus, it is imperative that the transmission algorithm is enhanced to ensure reliable communication through the network.
Understanding the dynamics of packet transmission on networks is also an important task in analyzing the onset of traffic congestion.From this viewpoint, Ohira et al. [17] illustrated the onset of phase transition behavior from a free-flow state to a congested state using square lattice-type network models.Arenas et al. [16] analyzed this phase transition behavior using hierarchical branching networks.Zhao et al. [13] showed that the heterogeneous property of a network inhibits traffic congestion in the case that the transmitting performance of each node is proportional to its degree.
On the other hand, the development of efficient routing methods that drastically reduce the congestion of a network presents another important task for carrying a large volume of data traffic.Recent studies concerning the design of routing strategies have evolved according to two basic ideas.The first is to select paths for transmitting packets based on only local information of the network, such as degree information or the number of packets waiting at adjacent nodes [4] [6] [8] [10] [11].Such routing strategies can be applied to networks of any size because it is not necessary to collect distance information for the whole network.The second idea is to utilize global information, such as shortest distance information for the network.For example, Echenique et al. [14] [15] proposed a deterministic routing strategy, which calculates efficient paths for the sending of packets by using distance information and dynamic information.Yan et al. [9] proposed a routing strategy that uses both distance and degree information for the network.Zhang et al. [7] proposed an adaptive routing strategy based on measuring the accumulated number of packets on shortest paths.A routing strategy incorporating traffic awareness was proposed by Wang et al. [5].Further, as an example of using global information, a routing method has been proposed based on mutually connected neural networks (MCNN routing) [21].This can be further enhanced by using reinforcement learning [22] or stochastic effects [23].The MCNN routing method searches all nodes adjacent to the sending node in order to route packets using the energy minimization principle, based on mutually connected neural networks.It is well known that mutually connected neural networks give an effective method for solving hard classes of problems, namely NP-hard problems.Furthermore, MCNN routing achieves good arrival rate for packets in regular topological communication networks.However, it performs poorly for irregular topological communication networks, because of the nonuniform distribution of packets [23].In order to improve its performance in this case, we have previously proposed a neural-based routing strategy that uses transmitting information [24].Concerning this strategy, it still remains to perform a performance evaluation regarding the alleviation of packet congestion in order to clarify how many packets the routing strategy can transmit in the case that the number of flowing packets increases.Such an evaluation is necessary if the routing strategy is to be employed in real communication networks.In addition, the congestion rate [16] can be used to determine the capability of avoiding packet congestion in a communication network model in which the number of packets to be transmitted continues to increase.This has been adopted in recent studies regarding the development of routing strategies [4] [6] [8] [10] [11].
From the above viewpoints, our neural-based routing method [24] is comprehensively evaluated in this paper using various performance measures such as the packet congestion rate, congestion levels, average arrival rate of packets, and average number of lost packets.The results show that the proposed neural-base routing method has a strong ability to carry large volumes of data compared with the conventional neural-based routing methods, and utilizes the transmitting information effectively.
This paper is organized as follows.In Section 2, we first describe how the routing method is constructed based on mutually connected neural networks, and then evaluate the MCNN routing method using the packet congestion rate.The modified version that utilizes transmitting information is then described in Section 3. Furthermore, our proposed neural-based routing method is evaluated in comparison to other conventional neural-based routing methods in Section 4. Finally, we conclude this paper in Section 5.

Routing Method Based on Mutually Connected Neural Networks
First, we explain how to construct the original MCNN routing method [21] because our proposed method is based on this.To proceed, we first describe the communication network model.Here, the communication network model is composed of a set of nodes and a set of links.Each node represents a host or a router, and each link represents a physical connection between two nodes.The node has a buffer to store packets, and can receive multiple packets simultaneously from its adjacent or connected nodes.The packets, whose sources and destinations are randomly determined, are then generated in the communication network.A packet is transmitted to a node according to the FIFO (first-in first-out) principle and is removed from the communication network if it arrives at its destination.
The communication network model has N nodes, where the ith node is connected to i N adjacent nodes.Next, we assign a mutually connected neural network to each node.In this MCNN routing method, each neuron in the neural network corresponds to a connection between the nodes.For example, the ijth neuron, which is the jth neuron in the ith neural network, represents the connection between the ith node and the jth node.Hence, a mutually connected neural network with i N neurons is assigned to each node.Figure 1 illustrates an example of the neural coding for the communication network model.
In the MCNN routing method, each node determines one of the adjacent nodes as the transmitting node at each iteration by minimizing the following energy function: where is the connection weight between the ikth neuron and the ilth neuron, l b is the buffer size of the lth adjacent node, l q is the number of packets stored in the lth adjacent node, l d is the shortest distance from the lth adjacent node to the destination of a packet at the ith node, c d is the longest distance among all shortest paths, η is the control parameter that decides the priorities of the first and second terms, and ξ is the control parameter that guarantees the uniqueness of the firing neurons.
In Equation ( 1), the second term expresses the load distribution of adjacent nodes, and the third term is the distance for a packet at the ith node to the destination.The last term guarantees that only a single neuron fires in each neural network.
The internal state of the mutually connected neural network is updated using the method proposed by Hopfield and Tank [25].The output of the ilth neuron il v is described as follows: if the th node is adjacent to the th node , 1 exp 0 otherwise .Moreover, when the ith and lth nodes are adjacent, the internal state of the ilth neuron il h changes according to the following differential equation: Note that Equation ( 3) is obtained by partially differentiating Equation ( 1) with respect to il v .In other words, From Equation ( 2) we obtain that Equation (5) indicates that the energy function E always decreases when il h changes according to Equation (3).In other words, the energy function of Equation ( 1) decreases according to the steepest descent method.
In this method, the firing of the neuron is determined using the threshold θ .That is, the ilth neuron fires if When the ilth neuron fires, a packet at the buffer head of the ith node is transmitted to the lth adjacent node.
The lth adjacent node of the ith node is selected as the transmitting node if it has the shortest distance to the destination of the packet and a smaller number of packets in its buffer among all of adjacent nodes.
As mentioned above, η in Equation (1) determines the priority of the load distribution of the adjacent nodes and the distance of packet to the destination.If 1 η = , then the transmitting packets are distributed widely throughout the network, in order to reduce heavily congestion at a node.However, the transmissions of almost all packets to their destinations will be delayed because in this case, the distance information for packets is ignored.On the other hand, if 0 η = , the packets are transmitted to their destinations along the paths with the shortest distance, with the result that many packets become trapped at nodes with large degree.
Next, we evaluate the effectiveness of the MCNN routing method.Because real communication networks have the scale-free property [26], we adopt the scale-free topology here as the computer network topology.We construct scale-free networks by the following procedure [26].We begin with a complete graph with 0 m nodes.Then, we insert a new node with 0 m links at every time step.Next, the links of the newly added node are connected to the existing nodes in the network with probability ( ) , where i k is the degree of the ith node ( ) , and N is the number of the nodes at the current iteration.For this study, we fixed 0 2 m = .
We conducted numerical experiments according to the following procedure.We generated p packets whose sources and the destinations are randomly determined at each iteration.Then, link selection is simultaneously conducted at every node, and the packets that are ready to exit each node are transmitted to adjacent nodes.A packet is removed from the network when it reaches its destination.In addition, the transmission of a packet is canceled if the buffer of the node to which the packet should be transmitted is full.
We set the number of iterations, T, to 3000, and the buffer size of each node, B, to 100.We set 3.0 β = , 0.1, 0.3, 0.5, 0.7 η = , and 0.9, and 0.3 ξ = in Equation (3), and set the threshold for the state of a neuron as 0.9 θ = .We conducted 10 simulations for five different values of η , and averaged the results.
We adopted the packet congestion rate [16] to evaluate the MCNN routing method.The packet congestion rate defined as follows: ( ) In Equation ( 6), C is the packet congestion rate, T is the number of iterations, and ( ) In Equation ( 7), ( ) n t is the number of packets at the tth iteration, N is the number of nodes, p is the number of packets generated at each iteration, and B is the buffer size of every node.
Figure 2 illustrates the packet congestion rate for the MCNN routing method.It can be seen that the packet congestion rate increases more quickly in the cases that 0.1 η = and 0.9 than for any others, because of the unbalanced setting between the distance to the destination and the distribution of the packets defined by Equation (1).On the other hand, with 0.5 η = this method avoids packet congestion, even if the number of generated packets increases.Hereafter, we set 0.5 η = for the MCNN routing method.To analyze why the MCNN routing method displays a poor congestion rate if the number of flowing packets increases, we next measure the congestion level of nodes in the computer network.To evaluate the congestion levels of the nodes, we employ a computer network model in which a fixed number of packets are flowing [23] [24] [28].As in the references [23] [24] [28], we introduce a buffer constraint and a packet moving constraint.Packets are then removed from the communication network if they are transmitted to nodes that have full buffers.In addition, packets are also removed when they exceed a fixed threshold for packet movements.If a packet arrives at its destination or is removed from the network, then we add an additional packet, to preserve the number of the packets flowing in the network.
In these simulations, we set the buffer size of every node to 100 and limited the number of packet movements between nodes to 20.The congestion level of the ith node at the tth iteration, ( ) In Equation ( 8), ( ) i q t is the number of queued packets of the ith node at the tth iteration, and B is the buffer size of every node.If ( ) i L t is close to 1, this indicates that the ith node has many packets waiting for further transmission.That is, the ith node is heavily congested.On the other hand, the ith node can afford to transmit packets to the next node quickly if ( ) i L t is close to 0. Figure 3 illustrates the congestion levels of the nodes for the MCNN routing method.In Figure 3, the congestion levels of the hub nodes with large degree clearly increase as the number of iterations increases.This indicates that the packets gradually accumulate at the hub nodes in the MCNN routing method.

Neural-Based Routing Method Using Transmitting Information
We have clarified that the conventional neural-based routing method [21] leads to congestion, because congestion at the hub nodes cannot be prevented.The reason why packet congestion increases for the convectional neural-based routing method may relate to the convergence to local minima of Equation ( 1) because only the shortest distance between a packet and its destination and the number of packets queued at adjacent nodes are considered while choosing an adjacent node for the routing of the packet.
One promising approach to overcome these problems is to consider all of the queuing information for every node.However, this requires heavy processing costs because such information constantly changes depending on the transmitting states of the packets.Another effective way might be to utilize limited queuing information for the nodes that lie on the shortest distance path between the packet and its destination.Hereafter, we refer to this limited queuing information as transmitting information.From this viewpoint, we propose a neural-based routing method that uses transmitting information by modifying the original energy function defined by Equation (1) as follows: ( ) Here, we define the subset of nodes ( ) l R t V ⊂ , whose elements are the nodes that lie on the shortest distance path to the destination passing through the lth adjacent node from the ith node at the tth iteration.That is, is the index of the jth node on the shortest distance path from the lth node at the tth iteration and ζ is the acquisition range of the transmitting information.Because the shortest distance path varies according to the destination node for each packet, the elements of ( ) l R t also change at each iteration.In Equation (9), lm b χ is the buffer size of the node lm χ , lm q χ is the number of packets queued at the node lm χ , l d is the shortest distance from the lth adjacent node to the destination node of a packet at the ith node, c d is the longest path length in the network, η is the control parameter that determines the priorities of the first and second terms, ξ is the control parameter that guarantees the uniqueness of firing neurons, ζ is the acquisition range of the transmitting information for transmission, and ( ) is a decaying parameter for the transmitting information.
In Equation ( 9), the first term expresses the transmitting information of nodes on the shortest distance path through the lth adjacent node, and the second term is the distance of a packet from the lth adjacent node to the destination.The third term guarantees that only one neuron fires in the neural network.If the distance to the destination of a packet from the lth adjacent node is the shortest among all of the adjacent nodes, and the transmitting information of the shortest distance path to the destination starting through the lth adjacent node is the lowest among all of the paths, then the lth adjacent node is selected as the transmitting node.
The cost of calculation for the conventional neural-based routing method becomes ( ) calculates the packet transmitting information between the jth and lth adjacent nodes as expressed by jl v in the second term in Equation ( 1).On the other hand, the cost for our routing method becomes ( ) ( ) O N O N α = because it only calculates packet transmitting information along the shortest distance paths, without using jl v .Thus, the proposed routing method then evaluates the optimum adjacent nodes much faster than the conventional neural-based routing method.

Performance Evaluation of the Proposed Method
We begin this section by evaluating our proposed routing method compared with the original MCNN routing method.Hereafter, we call the conventional MCNN routing method as the NN-O method in order to distinguish it from other neural-based routing methods.For these simulations, we adopted the same experimental assumptions and the same parameter settings as those in Section 2. We set 2.0 ζ = and 0.6 in Equation (9).
Figure 4 illustrates the packet congestion rate for both the NN-O and the proposed method.In Figure 4, the proposed method is seen to maintain a lower congestion rate than the NN-O method.Figure 5 illustrates the congestion levels of nodes for both the NN-O and the proposed method.In Figure 5, the proposed method can be observed to relieve packet congestion effectively at the hub nodes in comparison to the NN-O method.
Next, the performance of the proposed routing method is evaluated against three other conventional neural network based routing methods.The first is the original neural-based routing method (NN-O) [21], the second is a neural-based routing method incorporating the reinforcement learning (NN-R) [22], and the third is a neuralbased routing method that includes stochastic effects (NN-S) [23].
In the neural-based routing method with reinforcement learning [22], Equation ( 1) is replaced by the following energy function:  In Equation ( 10), ( ) η is updated according to the following algorithm: 1) ( ) for any i, where C is a constant that lies between 0 and 1.
2) The ith node transmits a packet at the tth iteration using 4) Go to 2.
In Equation (10), α is the learning rate, γ is the discount rate, and ( ) is the index of the node to which a packet is transmitted from the ith node at the t iteration.The function ( ) i r t in Equation ( 11) is defined by In this paper, we set the values of ( )( ) in Equation (10) to 0.5, and we set 0.1 α = and 0.9 γ = in Equation (12).These values are identical to those in reference [22].
In the neural-based routing method including stochastic effects [23], in Equation ( 10) is replaced by In these simulations, β and ξ are set to 3.0 and 1.0 respectively for all the methods, and we set in Equation ( 9). Figure 6 illustrates the packet rate for each of the routing methods.In Figure 6, the packet congestion rate for the NN-O, NN-R, and NN-S methods increases more quickly than for the proposed method.The congestion rate for the proposed method increases once the number of packets generated at each iteration increases to 10.Thus, the proposed method is able to inhibit the occurrence of the packet congestion as the number of packets flowing in the network increases.
Next, we evaluate the average numbers of arriving and lost packets, using a computer network model in which a fixed number of the packets flow [23] [24] [28].The average number of arriving packets, A , and the average number of lost packets, L , are defined by , , where a N is the number of arriving packets, g N is the number of generated packets, and l N is the number of lost packets.
Figure 7 illustrates the average numbers of arriving and lost packets for NN-O, NN-R, NN-S, and the proposed method.In Figure 7(a), the proposed method is seen to achieve a higher arrival rate for packets if the average number of packets at each node is between 10 and 40.In addition, the proposed method achieves the lowest lost packet rate of all the neural-based routing methods if the average number of packets at each node is less than 40.Thus, we have clarified that the proposed routing method achieves a better performance than the other neural-based routing methods if the number of packets flowing in the computer networks is small.The above results suggest that the transmitting information provides an effective method of avoiding packet congestion in order to reliably route packets.If the average number of packets at each node is larger than 50, then almost all of the nodes in the network will be holding more than 50% of the number of packets needed to fill their buffer.Thus, in this case packet congestion appears throughout the computer network, and any routing method will perform less effectively in removing congestion.

Conclusions
In this paper, we use the packet congestion rate to evaluate a proposed packet routing method that employs mutually connected neural network and transmitting information.The packet congestion rate is mainly employ-   ed to clarify performance in terms of congestion avoidance as the number of flowing packets gradually increases in a communication network.First, we showe that the original neural-based routing method is unable to alleviate packet congestion, because packets accumulate at hub nodes of large degree in the network, and packet congestion is then distributed into the network.To overcome this problem, we propose a neural-based routing method that exploits transmitting information, and evaluate our proposed method against conventional neuralbased routing methods, which incorporate either reinforcement learning or stochastic effects.From the results of the numerical simulations, we can conclude that our proposed method is able to reduce the packet congestion rate more effectively than the other neural-based routing methods.We also evaluate our proposed method in terms of the average numbers of arriving and lost packets by using a computer network model in which a fixed number of packets flow.We are able to confirm that our proposed method can maintain a higher arrival rate, or lower lost rate, than the other neural-based routing methods.From these results, we conclude that our proposed neural-based routing method achieves reliable communication by avoiding the congestion of packets through the effective use of transmitting information.

Figure 1 .
Figure 1.Schematic representation of the neural network coding in this paper.In this example, the ith node has three adjacent nodes.The neural network of the ith node is therefore constructed from three neurons that correspond to adjacent nodes 1, 2, and l.

Figure 3 .
Figure 3. Congestion levels of nodes for the MCNN routing method.In this figure, we fixed the number of flowing packets to 10.The node index is sorted in descending order of the degrees of the nodes.

Figure 4 .
Figure 4. Packet congestion rate for NN-O and our proposed method.

Figure 5 .
Figure 5. Congestion levels of nodes for (a) NN-O and (b) the proposed method.In this figure, we fixed the number of flowing packets to 10.The node index is sorted in descending order of the degrees of the nodes.

Figure 6 .
Figure 6.Packet congestion rates of the routing methods for the scale-free networks.

Figure 7 .
Figure 7. Relationship between the average number of packets at each node and (a) the average number of arriving packets and (b) the average number of lost packets.