The Stationary Distributions of a Class of Markov Chains

The objective of this paper is to find the stationary distribution of a certain class of Markov chains arising in a biological population involved in a specific type of evolutionary conflict, known as Parker’s model. In a population of such players, the result of repeated, infrequent, attempted invasions using strategies from   0,1,2, , 1 m   , is a Markov chain. The stationary distributions of this class of chains, for   3,4, , m   are derived in terms of previously known integer sequences. The asymptotic distribution (for ) is derived. m 


Introduction
In classical model in conflict theory [1], Parker's model [2], two individuals compete for a reward V by selecting times from some set.Here we suppose that the available times are integer values in .If a player chooses x and his opponent y then the payoff to the player , is given by The scenario envisaged is as follows.An individual choosing time x displays for that length of time, incurring a cost x.If x exceeds his opponents' choice y then he collects the reward.In the event of a tie the reward is shared.In a population in which individuals are restricted to play either u or v, where then the payoff matrix P is simply u the first row strictly dominates the second (that is 11 21 and 12 22 ), and if the second strictly dominates the first.We consider a population of individuals playing Parker's model.We suppose that the population evolves as fol-lows.Suppose at some time there is a population all of whom are playing a single strategy u (i.e. the population is monomorphic).A new strategy v arises by some random process.If u dominates v then the strategy v will be eliminated under any reasonable dynamic.On the other hand if v dominates u, it will rapidly increase in frequency and displace u.We will suppose that the introduction of new strategies is infrequent compared with the time taken for this replacement process.For a more detailed discussion of this model see [3].

The Class of Markov Chains
We investigate here the following class of Markov chains, [4], motivated by the above scenario.We suppose the available strategy set is and the reward The use of rather than V = 2 ensures that in every pair of strategies u and v, where  , one is dominant.The case allows particularly neat forms for the distributions, whereas other values of V require more complex, less elegant analysis and will be presented elsewhere.New strategies arise from the set M. If the current strategy is i and a new strategy j arises this latter will invade iff . If we suppose that the strategies arise with equal probabilities 1 m then we have a Markov chain with transition matrix Clearly this chain is irreducible.We investigate the stationary distribution of this class of Markov processess, for (the cases and . We derive a rational expression for these stationary distributions working throughout primarily in integers.For this reason we give the expression for the matrix below.

The Stationary Distribution
Now the dominant eigenvalue is m, and we derive a recurrence relation for the corresponding left eigenvector , the stationary distribution, where we set the right-most element equal to 1.It is straightforward to demonstrate that the final three elements of the eigenvector are , where, throughout, k is the identity matrix, k is a k element column vector and is a k element row vector.
and so We have This is valid for using and Suppose we write   y m for the sum of the elements is the stationary distribution of the Markov chain.We have immediately that  .This is sequence A001040 [5] specified as   a n , where our where the sequence is initiated with and We can extract individual elements of the stationary distributions.Suppose that is the i'th element of .Then we have that   u m 0 i   as one can see easily by considering the liniting case 1   , so the above expression tends to zero as and so . The sequence for is A058307, and is A058279 in [5].
We can apply the lemma immediately to the elements of the stationary distribution, expressed in the integer form.The ratios for 0 and 1 elements for , 1.307692, 1.308824, 1.308789, 1308789 illustrating the speed with which convergence takes place.We have no expression for the asymptotic value but for m = 200 the ratio is approximately 1.3087893731.The ratios for 1 and 2 elements for are 1, 0.5, 0.529412, 0.528090, 0.528131, 0.0.528130 and for m = 200 approximately 0.5281297672.
 Table 1 gives some values of .

 
, u m i

The Asymptotic Eigenvector
Having derived recurrence relations for the elements of the eigenvectors we now consider the limit as .We begin with a simple Lemma.

Lemma
In the absence of a simple way of evaluating the limiting ratios discussed above analytically we adopt a different method to derive the asymptotic stationary distribution, again expressed in integers.Suppose this is given by where is a constant, which depends on the initial values.
 , and

Proof
In a similar way we can obtain 3 , and so on.It is clear that the signs alternate.For ease we introduce the write we following notation; so that the sequences   i c and  for consist of positive integers.Similarly we write Now the denominator increases at a rate greater than x so the sequences and   i h consist of positive integers.Thus we have and so on, while ,   and so on.The theorem below gives recurrence relations for i , , c i d i g and , in terms of A058279 and A058307 [5].
  where   and where and .

Proof
We have that and , and we have already shown that , , , , which satisfy the formula given in the statement of the theorem.
We prove that if the formula holds for k X for then it holds for k for , and thence for Suppose then that the formula for k X holds for .Then since and substituting for the expressions given in the statement of the theorem we have Now clearly we also have so we have .Some idea of the speed of convergence can be gained by observing that these values agree with the elements of the eigenvector for except in the final 2 decimal places.15 m 

Conclusion
We have derived the stationary distribution of the frequencies of the available strategies in a population in which mutations occur infrequently, for Parke's model when the reward is 2 + and for integer valued strategies.These relate to certain known integer sequences.This work provides a base for further investigations for other values of the reward, and more complex invasion processes.

Discussion
Parker's model, which is also known as the Scotch Auction, is often used in the conflict theory literature as an example of a simple model in which there is no ESS (evolutionarily stable strategy).The implication of this is that there is no population assembly which is resistant to invasion.Of course if such a contests actually occurs it is important to ask what will happen in the population.This is the question which is addressed in [3], and which generates the class of matrices considered here.The stationary distribution then corresponds to the frequency with which one would observe a population to be playing a specific strategy, except if one happened on a population in transition.


The class of cases discusses above arises from Parker's model when we consider a fixed reward value 2 V   , and when the value of m, the range of possible strategies, is allowed to vary.It would be of interest to examine other possible values of V as m varies.For example, for , the "Markov matrices" have 1's for , 0's for and diagonal elements to make the row sums m.It is hoped to treat these models in a subsequent paper.
We observe from the numerical values that the most frequent strategy value played is 1, that the distribution is uni-modal and that the strategies  are played over 90% of the time; asymptotically approximately 0.90667 which agrees to five decimal places to the value for , while the mean value is asymptotically approximately 1.1207 which agrees to five decimal places to the value for .These latter figures confirm the rapidity of the convergence. z