^{1}

^{1}

^{2}

^{*}

^{1}

The original modified method of the direct delayed reaction has been used for the evaluation of food-obtaining strategy across spatial learning tasks in T-maze alternation. The optimal behavioral algorithms for each experimental day have been identified so that the animals obtain maximum possible food amount with minimal number of mistakes. Markov chain method has been used for the prognosis of rat’s behavioral strategy during the spatial learning task. The learning and decision-making represent the probabilistic transition process where the animal choice at each step (state) depends on the learning experience from previous step (state).

Both humans and animals live in a rich world of constantly changing external signals, thus the detection of changes and adequate reaction to environmental variation is crucial for the survival and successful adaptation. The learning and decision-making abilities are essential for adaptive behavioral strategy.

With no doubt, the behavioral studies remain the most prominent tool for exploring neural mechanisms of learning, memory and decision-making, though the past year experiments in behavioral and cognitive neuroscience [

The probabilistic and stochastic processes in cognitive neuroscience are considered as constitute foundations of learning and decision-making [

The question arises of how we go about understanding the probabilistic and stochastic processes that groundwork learning and memory. Due to a large number of possible parameters involved in learning, memory, and formation of adequate behavioral strategy, sometimes it seems difficult to generalize the results of behavioral studies. From this point, mathematical approach to the problem in general, and quantification of the measured parameters in particular, should be considered as the most reasonable means to identify the behavioral features and to interpret numeric data. Nowadays, such attitude is rather common to behavioral studies. Beside the wide range of traditional statistical methods used for analysis of behavioral parameters, different mathematical approaches and models are proposed for the data analysis in ethology, psychology, neuroscience, etc. [

Markov chains are one of the basic tools in modern statistical computing, providing the basis for numerical simulations conducted in a wide range of disciplines. This mathematical technique undergoes transitions from one state to another through a set of values which process can take. Every state depends only on the current state and not on the sequence of events that precede it. Markov chains have many applications in biological modeling [

There are several ways we may go while studying cognitive abilities. We can simply observe, record and analyze neurobiological, behavioral conformities, or/and make an attempt to construct quantitative models in order to understand the computations that underpin cognitive abilities. What different cognitive studies share is the attempt to identify the involvement of various brain states in the behavioral processes. Sometimes the segmenting of observed and measured behavioral processes into consequence elements is needed to explore and quantify transitions between them. Thorough inquiry of animal behavioral conformities across learning process gave huge body of information on behavioral consequences which can be tested with Markov chains analysis. Markov chains have shown to provide better estimation of learning conformities in comparison with other methods used to infer from behavior data treatment. Such modern approaches contribute in studying the cognitive abilities and their behavioral correlates.

From neurobiological point of view, it is interesting to perform extrapolation on the basis of experimental data in order to establish quantitatively the degree of learning and the dynamics of the memory. The results of behavioral experiments can be predicted by means of the mathematical model using one of the main objects of probabilistic-statistical investigation-Markov chains [

The experiments with using a modified method of direct delayed reactions made it possible to observe the learning process of the animals along with establishing the maximal delay and identifying an optimal algorithm of minimum errors and maximal reward. Here we will discuss the development of optimal algorithms and the dynamics of variability at delays of different duration.

31 albino rats of both sexes (with an average weight of 150 g) have been examined. The animals were individually housed in stainless steel cages in conditions of natural light-dark cycle and temperature of 20˚C ± 1˚C. The rats had free access to food and water throughout the whole experiment.

The rats were tested in a 25 cm-walled wooden T-maze (64 cm in length). The start compartment arm (47 cm in length) joined the two goal arms, each of which was 17 cm in width. Wooden feeders were situated at the far end of each goal arm. The floor under the feeders was electrified. The light and audio signal sources were attached to the top of starting compartment to study rat spatial learning through the different behavioral tasks using different experimental schemes (

The experiments were conducted on white rats using a modified method of direct delayed reactions [

Proposed approach gave us the possibility to characterize animal behavior and describe a learning algorithm [

Spatial-time Program | Pre-delayed Reactions | Delayed Reactions | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Time | The sequence of trials | Feeder with food | Intervals between trials in min | Duration of trial in sec | The type of trials: consistent (c) alternate (a) | Exit from the start compartment | The type of move | Food intake | The correction of move | Return to start compartment | Exit from the start compartment | The type of move | Return to start compartment |

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 |

11 - 00 | 0 | 1 | 2 | 5 | - | 0 | - | 1 | 1 | 0 | 1 | 1 | 0 |

1 | 1 | 3 | 5 | c | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | |

2 | 2 | 2 | 5 | a | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | |

3 | 1 | 5 | 5 | a | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | |

4 | 2 | 3 | 5 | a | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | |

5 | 2 | 4 | 5 | c | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | |

6 | 1 | 4 | 5 | a | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | |

7 | 1 | 2 | 5 | c | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | |

8 | 1 | 5 | 5 | c | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | |

9 | 2 | 3 | 5 | a | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | |

12 - 24 | 10 | 2 | 2 | 5 | c | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 1 |

feeder, where before delay it has got reinforcement (1). After this the experimenter returns the animal to the starting compartment (0).

The schematic diagram of behavioral strategies based on obtained algorithms for one of the experimental days is presented on

Description of Markov chain is as follows: we have a set of experimental trials (states)_{i} then it moves to state n_{i} at the next step with a probability denoted by P_{ij}, and it does not depend upon which states the chain was at before the current state. In Markov chain the probabilities P_{ij} are called transition probabilities. The process can remain in particular state, which occurs with probability P_{ij}. An initial probability distribution, defined on n, specifies the starting state. Usually this is done by specifying a particular state as the starting state of behavioral experiment. We present a behavioral method for estimating these functions.

The initial state of a system or a phenomenon and a transition from one state to another appear to be principal in the explanation of Markov chain. In our experiments the initial state is 0 or 1; while the transition may occur from 0 to 1, or vice versa from 0 to 0, or from 1 to 1.

For studying Markov chains it is necessary to describe the probabilistic character of transitions. It is possible

Trial # | Days | |||||||||
---|---|---|---|---|---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |

1 | 00000 | 11000 | 11000 | 11000 | 11001 | 11000 | 11111 | 11100 | 11110 | 11001 |

2 | 01000 | 10100 | 11000 | 10100 | 11000 | 11001 | 11001 | 11101 | 11001 | 10101 |

3 | 10010 | 11110 | 10100 | 11111 | 11001 | 11101 | 11001 | 10101 | 10111 | 10101 |

4 | 10010 | 11000 | 10100 | 11001 | 11001 | 11000 | 11000 | 11000 | 11001 | 10101 |

5 | 11000 | 11000 | 11001 | 10101 | 11001 | 11000 | 11000 | 11001 | 11001 | 11101 |

6 | 10100 | 11110 | 11110 | 11001 | 11001 | 11000 | 11000 | 11001 | 11111 | 11101 |

7 | 10100 | 10110 | 11000 | 11000 | 11001 | 10100 | 10100 | 10101 | 11001 | 10101 |

8 | 11000 | 11000 | 11100 | 11101 | 11001 | 10101 | 10101 | 11001 | 11001 | 10101 |

9 | 11000 | 11000 | 10100 | 11001 | 10101 | 11001 | 11001 | 11001 | 10101 | 11001 |

10 | 11000 | 11100 | 11101 | 11000 | 11001 | 11000 | 10100 | 11000 | 10101 | 11001 |

that the time intervals between transitions be permanent.

All states n (in our case_{ij} probability that the system will transit from state i to j does not depend on the type of behavior before state i. Proceeding from the explanation and features of the probability it is easy to assume that

For the modeling the above described experiment, it is convenient to define Markov chain as follows: let’s say ^{th} trial and does not depend on previous trials, one can say that Markov condition holds. The observed process is called “Markov chain”―random process with discrete (finite) state-space.

Let us assume that the observed behavioral process appears to be random chain with Markov properties, the possible values of which are 0 and 1, and the transition probabilities are determined (estimated) using obtained empirical frequencies. As the initial state is 0 or 1, the transition may occur from 0 to 1, from 0 to 0, or from 1 to 1. We consider the question of determining the probability that, given the chain is in state i, it will be in state j across behavioral treatment. Simply, if we will know the probability that the result of the first trial is 0, then we can define the probability that during the trial n^{th} result will also be 0

Let the conditional probability for transition from initial state 0 for trial n to state 1 in trial

It should be noted that the events

It is easy to determine the probability

where

Now we need to recall formulas:

which give recurrence equation for the reliable probability:

In the general case

In the case

Analogously, when

In the general case finding solutions to Equation (1) are very difficult, but in case when P_{n} does not depend on n, i.e.

The solution of stationary Equation (2) is simple:

As solution of stationary Equation (2) is known, we can get the general solution. We only have to suppose that

If we substitute (4) into the Equation (1) for P_{n}, we get:

From the stationary Equation (2) the following recurrence equation is correct

According to assumption (4) we get _{n}.

The Equation (5) can be used to predict the experimental results: if we know values for probabilities a, b and P_{1} for the first n number of trials, we can calculate the probability

The probabilities a, b and P_{1} can be estimated by corresponding empirical frequencies.

To illustrate this, let’s discuss in detail an algorithm of calculation of the probability (prognosis) ^{th} column (

1)

Analogically, the following results are obtained for 8^{th} - 14^{th} columns of

2)

3)

4)

5)

6)

7)

8)

In this paper, the original modified method of the direct delayed reaction on the basis of results of the Markov chains theory is developed. The proposed mathematical apparatus allow to calculate the probability (prognosis) _{1} for the first n number of trials are known. It should be noted that the probabilities of possible events of the theoretically calculated reactions coincide with the experimental data. It gives us an opportunity to use widely the above described method in neurophysiological investigations. In addition, it is possible to use them also for delayed reactions carried out by indirect and alternate methods. If we imagine everyday experimental results as one trial, it will be possible to make a long-term prognosis of the animals’ behavior.

Sulkhan N.Tsagareli,Nino G.Archvadze,OtarTavdishvili,MarikaGvajaia, (2016) The Prognosis of Delayed Reactions in Rats Using Markov Chains Method. Journal of Behavioral and Brain Science,06,19-27. doi: 10.4236/jbbs.2016.61003