On the Markov Chain Binomial Model

Rudolfer [1] studied properties and estimation of a   0,1 state Markov chain binomial (MCB) model of extra-binomial variation. The variance expression in Lemma 4 is stated without proof but is incorrect, resulting in both Lemma 5 and Theorem 2 also being incorrect. These errors were corrected in Rudolfer [2]. In Sections 2 and 3 of this paper, a new derivation of the variance expression in a setting involving the natural parameters   , , n p  is presented and the relation of the MCB model to Edwards’ [3] probability generating function (pgf) approach is discussed. Section 4 deals with estimation of the model parameters. Estimation by the maximum likelihood method is difficult for a larger number n of Markov trials due to the complexity of the calculation of probabilities using Equation (3.2) of Rudolfer [1]. In this section, the exact maximum likelihood estimation of model parameters is obtained utilizing a sequence of Markov trials each involving n observations from a   0,1 state MCB model and may be used for any value of n. Two examples in Section 5 illustrate the usefulness of the MCB model. The first example gives corrected results for Skellam’s Brassica data while the second applies the “sequence approach” to data from Crouchley and Pickles [4].


Introduction
Let be a stationary two-state Markov chain with states 0 (failure) and 1 (success), an initial probability distribution denoted in which and , and transition probabilities and represented in a transition probability matrix 1, 2, m   00 01 10 11 If is a sequence from a   , 1,2,    denotes the number of successes in n Markov trials.The random variable X may be said to have a Markov chain binomial (MCB) distribution with parameters n and p, and transition matrix P.
Rudolfer [1] studied properties and estimation for this state Markov chain binomial model.A formula for computing the probabilities is given as his Equation (3.2), and an expression for the variance of X is given as Equation (3.4) in his Lemma 4.
However, the variance expression is incorrect and hence Lemma 5 and Theorem 2 are also incorrect.These results were corrected in Rudolfer (2).Section 3 of this paper provides an alternative derivation of the correct versions of these results.

 
0,1 - , 0,1, , Even for relatively small values of n, computation of the probabilities   P X k  using Rudolfer's Equation (3.2) involves tedious summations making maximum likelihood estimation for the model difficult.Using the Edwards' [3] formulation and an approach introduced by Devore [5] and results given by Edwards [6], exact maximum likelihood estimation of model parameters is obtained in Section 4 using a sequence of observations assumed to come from a  -0,1 state Markov chain with the transition probability matrix P given in (1) above.
Two examples are presented in Section 5 to illustrate the effect of the corrections and of parameter estimation utilizing the sequence nature of the data.

The Markov Chain Binomial Model
Rudolfer [1] writes the matrix of transition probabilities For such a chain, the probability of a success at stage , , depends on k and the initial probability distribution q p .This is seen by noting that, for , Recursive application of this procedure results in  it follows that, as k , k tends to the limit independent of the initial state (0 or 1) and of the initial probability distribution   , q p .Also, if the initial probability distribution has then the transition matrix (2) can be written as q p p p P q q p q Replacing Rudolfer's [1] parameter  with r, the transition matrix (3) is seen to be equivalent to the one for the MCB model in Edwards [3].The parameter  in Rudolfer [1] has the same interpretation as that of r in Edwards [3], namely it is the first-order autocorrelation between 1 k Z  and k Z for .Thus, Rudolfer's and Edwards' MCB models are exactly the same probability model under re-parameterization with parameters n, p and 2 k   , and transition matrix P as given in (3).A random variable X with this distribution may be denoted by

 
, , ~, X MCB n p P  .Edwards' [3] approach involved the use of the probability generating function (pgf).For given n, the pgf of the MCB model is in which s is a dummy variable.Explicit expressions for the probabilities can be obtained from the pgf (4), although the calculations are not straightforward.For example, in the case of n = 3, the probabilities are  , 1,2, , These equations can also be obtained by substituting into the expressions for the probabilities as given by Rudolfer [1] at the beginning of Section 4.2 of his paper.

Rudolfer's [1,2] Corrected Results
Rudolfer's [1] Lemma 4 incorrectly expresses the variance of a random variable resulting in an incorrect statement of his Lemma 5 and thus Theorem 2. In Rudolfer [2] the variance expression is corrected and proper statements of Lemma 5 and Theorem 2 are given.A corrected version of Lemma 4 is presented with an alternative derivation, and is followed by the corrected Theorem 2.

Corrected Lemma 4 of Rudolfer [1]
For a random variable Proof: The n-th step transition probability matrix is equal to as is easily shown by diagonalizing the transition matrix P given in (3), using its right and left eigenvectors and its eigenvalues.For where .
  which immediately gives the result in the theorem.

Estimating the Model
Consider N trials from a   0,1 -state Markov chain with parameters n, p and  .Let and This nonlinear equation can be solved for  by iteration using MAPLE or other mathematical software.

Maximum Likelihood Estimation of p and  for Sequence Data
Even for a relatively small value of n, maximum likeli-hood estimation of the model parameters p and  may prove to be difficult using either Rudolfer's [1] Equation (3.2) (in this case, the summation in the expression for the probabilities is complex) or Edwards' [3]  The likelihood function for such sequence data can be written in the form in which 1 is the number of the N sequences that begin with the occurrence of state 1 (that is, success), is the number of sequences that begin with the occurrence of state 0 (that is, failure), and ij denotes the total number of transitions from state i to state j observed within the N sequences.Since each of the N sequences involves n trials, a total of nN transitions will be observed.If the transition probabilities are written in terms of p, N and  as in the version of the transition matrix P in expression (3), the likelihood function (10) becomes Devore [5] obtained maximum likelihood solutions by ignoring the contribution of the initial state 1 Z , and considering only the modified likelihood function The solutions obtained restricting attention to this modified likelihood function are Equating expressions (15) and ( 16), and letting , which can be solved by standard methods.Substituting the solution from (17) into either (15) or ( 16) provides the solution p  .

Application to Skellam's [7] Brassica Data
Rudolfer [1,2] used the "Brassica" data of Skellam [7] to illustrate the model and parameter estimation.Using this data and the method of moments with result (5) of the corrected Lemma 4, the obtained estimate ˆ0.5806 p  is essentially the same as the maximum likelihood estimate .The corrected moment Equation (9) for the variance in the case of n = 3 is On substitution of , the quadratic Equation (18) gives the possible root , rejecting the other root as it is less than .
Comparisons of the p-values and of the values of the Chi-square statistics given in Table 1 suggest that the Markov chain binomial model fitted by the method of moments using the corrected variance Formula (5) gives a fit very similar to those provided by the maximum likelihood method in Rudolfer [1] and the corrected method of moments method in Rudolfer [2].

Application to Human Birth-Order Data
Using the solutions provided by Devore's [5] modified likelihood function and by Edwards' [6] full likelihood function, the MCB model is fitted to a case in which a sequence of observations is given.Table 2 gives birthorder data for families with four children from Finland and from the United States.This data appears in Crouchley and Pickles [4].It is assumed that the data are generated by a Markovian dependent sequence as defined in (1).The parameters p and  are the quantities of interest in the analysis.
Let state 0 refer to females and state 1 to males.For the Finish data, the full likelihood function, as given by (10) and then (11), is

Discussion
In the application of the Markov chain binomial model to the Brassica data of Skellum [7] using the method of moments, Table 2 illustrates that the variance expression (5) results in a fit little different from that provided by the method of moments using Rudolfer's [2] corrected values.
The second example dealing with birth-order data presented in Crouchley and Pickles [4] illustrates the usefulness of the "sequence data approach" in finding maximum likelihood estimates for the MCB model.Very little difference is seen in the estimates in this approach between using the full likelihood equation solution discussed by Edwards [6] and using the modified likelihood equation as discussed by Devore [5].
From the variance expression (5), let

4 . 1 .
 N are independent, identically distributed random variables each having the MCB distribution with parameters n, p and  .Moment Estimation of p and  sample mean and sample second raw moment respectively.Using the method of moments, estimators and p  are determined as solutions to the equations 0 np x   provides a solution using the full likelyhood function (11).Equating each of the partial derivatives log L p   and log L    of the log-likelihood function to zero and solving for  provides the two expressions

Table 2 . Frequencies of birth-order for families of four child- ren from Finland and the United States.
Similarly, for the case of the United States population the estimated parameters from the full likelihood func-