Convergence of Invariant Measures of Truncation Approximations to Markov Processes

Let Q be the Q-matrixof an irreducible, positive recurrent Markov process on a countable state space. We show that, under a number of conditions, the stationary distributions of the n × n north-west corner augmentations of Q converge in total variation to the stationary distribution of the process. Twoconditions guaranteeing such convergence include exponential ergodicity and stochastic monotonicity of the process. The same also holds for processes dominated by a stochastically monotone Markov process. In addition, we shall show that finite perturbations of stochastically monotone processes may be viewed as being dominated by a stochastically monotone process, thus extending the scope of these results to a larger class of processes. Consequently, the augmentation method provides an attractive, intuitive method for approximating the stationary distributions of a large class of Markov processes on countably infinite state spaces from a finite amount of known information.


Introduction
Let     ., , Q q i j i j S  atrix  be the stable, conservative of a continuous-time Markov process on a countable state space The satisfies In addition, we assume that Q is regular, which means there exists no non-trivial, non-negative solution for some (and then all) .0 k  Under these assumptions, the state transition probabilities of the process are given by the unique Q-function which satisfies the Kol-mogorov backward equations,    , , , , 0 The object F , which is also called a transition function, is a family of matrices indexed over the reals which constitutes an analytic semi-group.an analytic semi-group is characterised by three properties: S S  0 F is the identity matrix, the row sums of t F are less than or equal to unity and s t F  is equal to the matrix product s t F F for all .This last property, known as the Chapman-Kolmogorov equation, implies , s t  0 . Thus, even though F t is generally thought of as the matrix of state transition probabilities at time t, it serves as an analogue to the t-th power of the transition matrix of a discrete-time Markov chain on the state space Consequently, using the superscript to denote as a function of t should not cause any confusion.While on the subject of notation, we should mention that we are using a standard notation common in the literature of continuous-time Markov processes on general state spaces.In the discrete state space setting, this notation causes matrices to look like functions of two variables (or kernels) while measures and vectores appear to be functions over the state space.We have elected to follow this notation in an endeavour to reduce the number of subscripts and superscripts in the sequel.

S F Q
Note that in the conservative setting posed here, regularity of is equivalent to honesty and uniqueness of the transition function, that is,

 
, j 1 .For a positive recurrent process, it can be shown (for example, see Theorem 5.1.6 in [1]) that the More generally, any measure satisfying (1) is called an invariant or stationary measure for the process.If, in addition, the measure has mass 1, it is referred to as a stationary or invariant distribution.Any measure satisfying (1) with " " replaced by "  " is called a subinvari- ant measure for F .Conversely, if F has a stationary distribution , then the process is positive recurrent and .
 In this paper, we are interested in approximating using the n north-west corner truncations of .The analogous problem for discrete-time Markov chains has been studied in [2][3][4][5][6][7].The final reference contains a review of the literature on the discrete-time version of the truncation problem.Some properties of truncation in continuous-time Markov processes were studied in [8,9].
The truncation ( ) n is not conservative.By adding the discarded transition rates to , we may produce a  which generates a unique, honest, finite, continuous-time Markov process.For example, we may choose to perform linear augmentation, where the aggregate of the transition rates outside of n is dispersed amongst the states in according to some S  probability measure .Then, the An important example of this is where we only augment a single column, say , in which case   n h  is the Dirac measure at h and we obtain The order augmentation as


Here, jh  denotes the kronecker delta.
 n will be its invariant probability measure.Two obvious questions now arise.Firstly, when does Here, we use w   to denote convergence in total variation norm.Secondly, how quickly does this convergence occur?This paper considers the first question.We shall present augmentation strategies for approximating invariant distributions for two classes of Markov processes via for n large.The classes are: for some 0   .Such processes are called exponentially ergodic. Stochastically monotone Markov processes, which have the property that for all , , , and 0 i k i k n S t    , and processes dominated by stochastically monotone processes.Parallelling results for discrete-time chains in [7], we shall also show that Markov processes constructed from finite perturbations of stochastically monotone processes are always dominated by some other stochastically monotone process.This extends the class of processes for which our results are applicable.
In the next section, we begin by showing that the limit of the   π n  is unique when it exists.Then, Section 3 considers exponentially ergodic Markov processes while Section 4 studies stochastically monotone Markov processes and their above-mentioned variations.
Finally, some concluding remarks are made in Section 5.

Preliminaries
The problem of proving that   π w h n π   may be bro- ken into two parts.Firstly we must show that   converges weakly to some limit, say π h n π , and secondly, that π π  .We consider the latter in this section.
Theorem 2.1 Consider a sequence of linearly augmented derived from Q and let Proof: Let   n F denote the minimal -function.
 and .This can be seen inductively using the backward integral recurrences for   n 0 t  F and   h n F .The argument parallels the proof of Theorem 2.2.14 in [1] which states that , ,

for
, where , , 0, Applying (4) to (6) together with monotone convergence shows that   t n i  monotonically decreases to 0 as .Taking limits in n on both sides of (5) then completes the proof.
for .Positive recurrence establishes anequivalence between the stationary distributions for   , 0 n F and invariant distributions for .An invariant distribution for an arbitrary is any probability Let us assume for the moment that converges   π h n weakly to some limit measure  , We require that π π  .Weak convergence to π implies that π is a probability distribution.By taking the limit infimum on both sides of (7) and applying Fatou's Lemma, we have The measure π is therefore a subin- variant probability measure for .However, is positive recurrent and hence, by Theorem 4 in [10], π is both invariant and the unique probability measure satisfying (1).Hence, π π  .

Exponential Ergodicity
Let Q be the of a positive recurrent Markov process   as where is taken to be   0 n , the invariant distribution for   0 n .In order to do this, we shall require the notion of a -norm.Let  .In future, we abreviate this to .The Vnorm of a signed measure n is then Rather than working with the Q-matrix augmentationsdirectly, we will use the  -resolvents associated with these.The  -resolvent of a continuoust-time Markov process is the stochastic matrix


We note that R  satisfies the resolvent forms of both the backward and forward equations which are ij and tegrating both sides of ( 5) as where . Part ii then follows by taking limits in (8) and ng that ned analo-, Petite go exist co sets for discrete-time chains are defi there usly.According to Theorem 5.1 in [11], the following three drift conditions are equivalent, although the petite set C and function V may differ in each instance.

 
An irreducible continuous-time Markov process X is ke V -uniformly ergodic if, for some invariant probability rnel π, π 0 as In the special case where 1 V  , the chain is said to be uniformly ergodic or strongly ergodic: For all ii.For all 0 T  , the T ke godic; iii.For 0 In particular, it is T V -uniformly erg V  -uniformly ergodic and V  -iformly ergodic where , , and ii  iii  iv.This was proved in Theorem 5.3 of [11].i  iv.T eorem 5.1 of [11] X is -uniformly ergodic for some th ws tha an ap en to is the -skeleton for any 0 T  and plication of Theorem 16.0.1 in [13] sho t so for some M   and . Geometric ergodici skeleto ty of the T -n Tn F then fol lows from the definition of -norm o is 3.
exist constants (9) Without loss of generality, we may take the V .Next, suppose that the Mark v process X exponenttially ergodic.From Theorem 2, there 0 , c d   and a function 1

can always and assume that
The state spac be re- 1  for labelled to accommodate this convention.Then, since Multiplying both sides by   ,0 n R  and re-arranging, we obtain as noted in the pr where ,0 for all large enough., set It can b onditions of the theorem are satisfied and some V in Th orem 6.1 of [14] e seen that the c so there exists 1   such that   where and where is the unique invariant distribution for by  and  .Note that this is true for every n N  so that the rate of convergence  is independent of uncation size.In addition, by applying the precedin argument directly to Q instead of  0 Q , we also have g the tr by assumption, (9) holds and

V-uniform
same converge and   ,0 n R  not only ar R  ly ergodic, they are geometrically ergodic with the nce rate  .
We can now prove the main result of this section.Theorem 3.3 Let X be an exponentially ergodic, contin ible st uous-time Markov chain on a countable, irreduc ate space S .Let π and ( ) 0 π n be the invariant distributions for Q and   0 n Q respectively.Then, Proof: Cho e an ar ry number 2 m  .From  os bitra the triangle inequality, we have .
As was pointed out in [7], if A and are two sto-B chastic matrices, then for .Applying this to the last term in (13), we obtain ( 14) where  , we can use domin ude ird term in (13) vanishes as n tends to infinity.Thus, ated convergence to concl that the th for and since m was chosen arbitrar y, m   , il ,if 0, 0, otherwise, The process with this -matrix is essentially a renewal with renewal time Q s of process s marked by visits to state 0. Each renewal time consists of a geometric number of exponential time mean 1  followed by an exponential time of mean 1  .At each jum , the process passes from state with probability . Existence of the stationary distribution ensures positive recurrence.
Next, let the vector V  be given by     , where c is a positive number.Then, the drift condition  , from which it is evident that for all n   , in which case we write    .If F and F  are two transition functions, we say that F  stochastically dominates F (written A more strict classification is stochastic comparability.The transition functions F and F  are stochastically comparable if We use the notati n o F F   that F to mean and F  are stochastically comp arable.A stica monotone Markov process is one whose transition function is stochastically comparable to itself.Thus, if stocha lly F is stochastically dominated by a transition function F  which itself is stochastically monotone, then F and F  are stochasticallly comparable.Clearly,  eorem is th y to obtaining sufficien onditions for (2) to hold in continuous time.It characterises stochastic comparability and monotonicity in terms of Q -matrix structure and is a special case of a more general result which was proved in [15] (also see Theorem 7.3.4 in [1] for an account).The reader is drected to the last two citations for the proof.
ii.Let Q be a conservative Q -matrix. , We also extend and to by appending a countably infinite number of 0's to each, so that for all Note that riant fo esp. .
positive recurrent on some subset of   0,1, , n  , S the m containing n and transient elsewhere in easure is the limiting distribution of en given an appriate initial distribution.The stochastically monotone m .
e π is the unique stationary distribution for .F The sequence     π e tight and so is therefor and so is at least as good a ed no in As was poin [7], intwi res on a c ble se nce in tal variation.We therefore have the fo Theorem 4. to be its invariant distribution.Then, which is the optimal approximation in its tail mass more closely approximates that of π .

Pro
The of: fact that, for all Then, we The analogous statement holds for and the proof is completed by letting first n and th n k tend to infinity.
As remarked in [6]  Q concerning the truncation approximation of invariant distribu ns of birth-death processes and su itical Markov branching processes, for example, see [16][17][18].The convergence of quasistationary d tions of truncations to the invariant distribution of th riginal ess also holds under the weaker conditions we disin th xt two subsections.

Proc Dominated by Stochastically Monotone Proces
Now we shall consider a much larger class of Markov processes, namely those whose transition functions are stochastically dominated by a positive recurrent, stochastically monotone process.To begin, let F  be the stochastically monotone transition function of an irreducible, positive recurrent Markov process.Suppose that Convergence in total variation ollows in the a as in the proof of Theorem 4. and the same is true of

Perturbed Monotone Markov Processes
Finally, we consider an even more general class of ko s intro v processes which wa duced in [7].We say is a finite perturbation Q if the two Q-matrices differ in at most a finite number of co be Q ically monotone and suppose without loss of ge- Observe that is stochastically monotone.This is due firstly to the way in which th ave been construct ,

Example
Conrth-death process, whose tridiagonal -matrix   .It remains to study the speed of such convergence.An understanding of the convergence rate would enable the truncation size to be selected in order to guarantee that the measure   n π approximates π to a de- sired degree of accuracy.

Acknowledgements
will play an important role in allowing us to pass between the continuous-time process and the discretetime  -resolvent chain.The drift conditions require the notion of a petite set in both continuoustime processes and discrete-time chains.Let   S  denote the Borel  -algebra on S.Then, A set Theorem discussion.So too was the optimality of as n   was established in   π n n n approximation to π.To prove convergence in total variation, fix an arb obtain itrary finite .k S  i ariant distribution for Q and denote the invariant distributions of arbitrary n n  north-west corner augmentations   Chains: An ach," r Series in Sta-M) Basal CONICYT Program PFB 03 and YT grant 1070344.AGH would like to thank Servet Martinez for interesting discussion on the truncation of stochastically monotone processes.AGH dedicates this article to co-author Richard Tweedie, who passed away after this work was started.
Consider an increasing sequence of in addition to R  (18)h-west corner truncations of Q and Q  augmented in the n th column.By extending these in the analogous way to(18)and applying Part i of Theorem 4.1, we see that   n n F and   n n F  are stochastically comp able.
Q ond lier, F must be positive recurrent and the invariant ns π and π  , corresp F and F  respectively, satisfy π π   .Let   n n Q and   n n Q  respectively denote the n n  π n corresponding to the augmentation   n