An Infinite Series Expression for the Joint Probability Density Function of OccupationTimes in a Three-State Markov Chain

James Evans, Andrzej Korzeniowski

Department of Mathematics, University of Texas at Arlington, Arlington, TX, USA.

DOI: 10.4236/apm.2025.1511038                

Abstract

We obtain an infinite series expression for the joint probability density function of occupation times in a Three-State Markov chain. By extending the identity given by Darroch, we demonstrate that Pedler’s technique of employing moment generating functions can be applied to the three-state case as well.

Keywords

Occupation Times, Markov Chains, Moment Generating Functions, Probability Density Function

1. Introduction

The occupation time probability density function was the subject of several research papers in the 60s and 70s as seen in Darroch [1], Pedler [2], Good [3], and Hsia [4]. More recently, there has been renewed interest in the subject as seen in Sericola [5] as well as applications in Fang [6] and Guo [7]. In this paper, we extend Darroch’s identity to any finite state Markov chain and then derive an explicit three-state joint probability density function in a manner similar to Pedler’s original two-state derivation.

2. Occupation Time for a Three-State Markov Chain

Consider an irreducible continuous time Three-State Markov chain $M\left(t\right)$ with rate matrix $R=\left({\lambda }_{ij}\right),{\lambda }_{ij}>0$ if $i\ne j$ ranging over states $E_1,E_2,E_3$ with initial distribution ${\Pi }_3=\left[{\pi }_1,{\pi }_2,{\pi }_3\right]$ where $0\le {\pi }_i\le 1$ denotes the probability of starting in $E_i$ at time 0. Then for $0\le u\le t$ , the occupation times of $E_1,E_2,E_3$ by time $t$ are

$X\left(t\right)={\displaystyle {\int }_0^t{1}_{E_1}\text{d}u},Y\left(t\right)={\displaystyle {\int }_0^t{1}_{E_2}\text{d}u},Z\left(t\right)={\displaystyle {\int }_0^t{1}_{E_3}\text{d}u}$

where $X\left(t\right)+Y\left(t\right)+Z\left(t\right)=t$ and ${\pi }_1+{\pi }_2+{\pi }_3=1$ .

Theorem 1 Given $X\left(t\right)=x,Y\left(t\right)=y,t>0$ , the joint probability density function of the occupation time for a Three-State Markov chain up to time t is given by

$f\left(x,y,t\right)={\text{e}}^{-x\left({\lambda }_{12}+{\lambda }_{13}-{\lambda }_{31}-{\lambda }_{32}\right)-y\left({\lambda }_{21}+{\lambda }_{23}-{\lambda }_{31}-{\lambda }_{32}\right)-t\left({\lambda }_{31}+{\lambda }_{32}\right)}g\left(x,y,t-y-x\right)$

where

$g\left(x,y,t-y-x\right)={\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=1}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l}{y}^{m-k-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m-l\right)!\left(m-k-1\right)!\left(m-j-1\right)!}$

$+{\pi }_1{\displaystyle \sum _{m=2}^{\infty }{\displaystyle \sum _{k=1}^{m-1}{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)\frac{x^m{y}^{m-k-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m-k-1\right)!\left(m-j-1\right)!}$

$+{\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{j=0}^{m-1}b}}\left(m,0,0,j\right)\frac{x^m{y}^{m-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m-1\right)!\left(m-j-1\right)!}$

$+{\pi }_1{\delta }_y{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,m,0\right)\frac{x^m{\left(t-x-y\right)}^{m-1}}{\left(m\right)!\left(m-1\right)!}$

$+{\pi }_1{\delta }_z{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,0,m\right)\frac{x^m{y}^{m-1}}{\left(m\right)!\left(m-1\right)!}$

$+{\pi }_1{\delta }_y{\delta }_z$

$+{\pi }_2{\displaystyle \sum _{m=2}^{\infty }{\displaystyle \sum _{l=1}^{m-1}{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l-1}{y}^{m-k}{\left(t-x-y\right)}^{m-j-1}}{\left(m-l-1\right)!\left(m-k\right)!\left(m-j-1\right)!}$

$+{\pi }_2{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{k=1}^m{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)\frac{x^{m-1}{y}^{m-k}{\left(t-x-y\right)}^{m-j-1}}{\left(m-1\right)!\left(m-k\right)!\left(m-j-1\right)!}$

$+{\pi }_2{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{j=0}^{m-1}b}}\left(m,0,0,j\right)\frac{x^{m-1}{y}^m{\left(t-x-y\right)}^{m-j-1}}{\left(m-1\right)!\left(m\right)!\left(m-j-1\right)!}$

$+{\pi }_2{\delta }_x{\displaystyle \sum _{m=1}^{\infty }b}\left(m,m,0,0\right)\frac{y^m{\left(t-x-y\right)}^{m-1}}{\left(m\right)!\left(m-1\right)!}$

$+{\pi }_2{\delta }_z{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,0,m\right)\frac{x^{m-1}{y}^m}{\left(m-1\right)!\left(m\right)!}$

$+{\pi }_2{\delta }_x{\delta }_z$

$+{\pi }_3{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=0}^{m-1}{\displaystyle \sum _{k=0}^{m-l-1}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l-1}{y}^{m-k-1}{(t-x-y)}^{m-j}}{\left(m-l-1\right)!\left(m-k-1\right)!\left(m-j\right)!}$

$+{\pi }_3{\displaystyle \sum _{m=2}^{\infty }{\displaystyle \sum _{l=1}^{m-1}b}}\left(m,l,m-l,0\right)\frac{x^{m-l-1}{y}^{l-1}{\left(t-x-y\right)}^m}{\left(m-l-1\right)!\left(l-1\right)!\left(m\right)!}$

$+{\pi }_3{\delta }_x{\displaystyle \sum _{m=1}^{\infty }b}\left(m,m,0,0\right)\frac{y^{m-1}{\left(t-x-y\right)}^m}{\left(m-1\right)!\left(m\right)!}$

$+{\pi }_3{\delta }_y{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,m,0\right)\frac{x^{m-1}{\left(t-x-y\right)}^m}{\left(m-1\right)!\left(m\right)!}$

$+{\pi }_3{\delta }_x{\delta }_y$

$+c_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=0}^{m-1}{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l-1}{y}^{m-k}{\left(t-x-y\right)}^{m-j}}{\left(m-l-1\right)!\left(m-k\right)!\left(m-j\right)!}$

$+c_1{\delta }_x{\displaystyle \sum _{m=1}^{\infty }b}\left(m,m,0,0\right)\frac{y^m{\left(t-x-y\right)}^m}{\left(m\right)!\left(m\right)!}$

$+c_1{\delta }_x$

$+c_2{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=1}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l}{y}^{m-k-1}{\left(t-x-y\right)}^{m-j}}{\left(m-l\right)!\left(m-k-1\right)!\left(m-j\right)!}$

$+c_2{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{k=0}^{m-1}{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)\frac{x^m{y}^{m-k-1}{\left(t-x-y\right)}^{m-j}}{\left(m\right)!\left(m-k-1\right)!\left(m-j\right)!}$ (1)

$+c_2{\delta }_y{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,m,0\right)\frac{x^m{\left(t-x-y\right)}^m}{\left(m\right)!\left(m\right)!}$

$+c_2{\delta }_y$

$+c_3{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=1}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l}{y}^{m-k}{\left(t-x-y\right)}^{m-j-1}}{\left(m-l\right)!\left(m-k\right)!\left(m-j-1\right)!}$

$+c_3{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{k=1}^m{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)\frac{x^m{y}^{m-k}{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m-k\right)!\left(m-j-1\right)!}$

$+c_3{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{j=0}^{m-1}b}}\left(m,0,0,j\right)\frac{x^m{y}^m{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m\right)!\left(m-j-1\right)!}$

$+c_3{\delta }_z{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,0,m\right)\frac{x^m{y}^m}{\left(m\right)!\left(m\right)!}$

$+c_3{\delta }_z$

$+c_4{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=0}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l}{y}^{m-k}{\left(t-x-y\right)}^{m-j}}{\left(m-l\right)!\left(m-k\right)!\left(m-j\right)!}$

$+c_4$

$\begin{array}{l}{c}_1={\pi }_2{\lambda }_{23}+{\pi }_3{\lambda }_{32}\\ c_2={\pi }_1{\lambda }_{13}+{\pi }_3{\lambda }_{31}\\ c_3={\pi }_1{\lambda }_{12}+{\pi }_2{\lambda }_{21}\\ c_4={\pi }_1\left[{\lambda }_{12}{\lambda }_{13}+\left({\lambda }_{13}-{\lambda }_{23}\right)\left({\lambda }_{32}-{\lambda }_{12}\right)\right]+{\pi }_2\left[{\lambda }_{21}{\lambda }_{23}+\left({\lambda }_{31}-{\lambda }_{21}\right)\left({\lambda }_{23}-{\lambda }_{13}\right)\right]\\ +{\pi }_3\left[{\lambda }_{31}{\lambda }_{32}+\left({\lambda }_{12}-{\lambda }_{32}\right)\left({\lambda }_{31}-{\lambda }_{21}\right)\right]\end{array}$

$\begin{array}{l}b\left(m,l,k,j\right)\\ =\left(\begin{array}{c}m\\ l\end{array}\right)\left(\begin{array}{c}m-l\\ k\end{array}\right)\left(\begin{array}{c}m-l-k\\ j\end{array}\right){\left({\lambda }_{23}{\lambda }_{32}\right)}^l{\left({\lambda }_{13}{\lambda }_{31}\right)}^k{\left({\lambda }_{12}{\lambda }_{21}\right)}^j{\left({\lambda }_{12}{\lambda }_{23}{\lambda }_{31}+{\lambda }_{13}{\lambda }_{21}{\lambda }_{32}\right)}^{m-l-k-j}\end{array}$

and $\delta \left(x\right)={\delta }_x,\delta \left(y\right)={\delta }_y,\delta \left(t-x-y\right)={\delta }_z$ are Dirac delta functionals satisfying the property that ${\displaystyle {\int }_{\alpha }^{\beta }\delta \left(u-k\right)f\left(u\right)\text{d}u}={\displaystyle {\int }_{\alpha }^{\beta }\delta \left(k-u\right)f\left(u\right)\text{d}u}=f\left(k\right)$ if $k\in \left[\alpha ,\beta \right]$ and 0 otherwise.

3. Derivation of Three-State Occupation Time Probability Density Function

The proof of Theorem 1 is based on the following Lemmas:

Lemma 2

$E\left[{\text{e}}^{-{\theta }_{n-1}\cdot X_{n-1}\left(t\right)}\right]={\Pi }_n{\text{e}}^{t\left(R-H\right)}{1}_n$ (2)

where ${\theta }_{n-1}\cdot X_{n-1}\left(t\right)$ is the dot product of ${\theta }_{n-1}=\left[k_1{\theta }_1,k_2{\theta }_2,\cdots ,k_{n-1}{\theta }_{n-1}\right]$ and $X_{n-1}\left(t\right)={\left[X_1\left(t\right),X_2\left(t\right),\cdots ,X_{n-1}\left(t\right)\right]}^{\text{T}}$ , ${\Pi }_n=\left[{\pi }_1,{\pi }_2,\cdots ,{\pi }_n\right]$ ,$1_n$ the column vector of 1s, $X_i\left(t\right)={\displaystyle {\int }_0^t{1}_{E_i}\text{d}u}$ the occupation time of state $i$ by time $t>0$ , and

$H=\left[\begin{array}{cccc}{k}_1{\theta }_1& 0& \cdots & 0\\ 0& \ddots & \ddots & ⋮\\ ⋮& \ddots & k_{n-1}{\theta }_{n-1}& 0\\ 0& \cdots & 0& 0\end{array}\right]$

Proof of Lemma 2.

From Darroch [1], one may write the moment generating function of the weighted sum of states as

$E\left[{\text{e}}^{-\mu Y\left(t\right)}\right]={\Pi }_n{\text{e}}^{t\left(R-\mu D_1\right)}{1}_n$

with $D_1=\left[\begin{array}{cc}D& 0\\ 0& 0\end{array}\right]$ , $D$ the matrix with elements ${\delta }_{ij}{d}_i$ ,$Y\left(t\right)={\displaystyle \sum _{i=1}^s{d}_i}{X}_i\left(t\right)$ where $d_i$ are arbitrary weights, ${\delta }_{ij}$ the Kronecker delta, and $1\le s<n$

Then setting $d_i=\frac{{\theta }_i{k}_i}{\mu }$ , with $k_i$ arbitrary, the moment generating function is extended to a multivariate moment generating function and we obtain (2) as desired. $\square$

Lemma 3

The inverse Laplace transform of the expression below reads

$\begin{array}{l}{ℒ}^{-1}\left[{\pi }_{1}vw{\displaystyle \sum _{m=0}^{\infty }{\displaystyle \sum _{l=0}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)u^{-m+l-1}{v}^{-m+k-1}{w}^{-m+j-1}\right]\left(x,y,t-y-x\right)\\ ={\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=1}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)\frac{x^{m-l}{y}^{m-k-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m-l\right)!\left(m-k-1\right)!\left(m-j-1\right)!}\\ +{\pi }_1{\displaystyle \sum _{m=2}^{\infty }{\displaystyle \sum _{k=1}^{m-1}{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)\frac{x^m{y}^{m-k-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m-k-1\right)!\left(m-j-1\right)!}\\ +{\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{j=0}^{m-1}b}}\left(m,0,0,j\right)\frac{x^m{y}^{m-1}{\left(t-x-y\right)}^{m-j-1}}{\left(m\right)!\left(m-1\right)!\left(m-j-1\right)!}\\ +{\pi }_1{\delta }_y{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,m,0\right)\frac{x^m{\left(t-x-y\right)}^{m-1}}{\left(m\right)!\left(m-1\right)!}\\ +{\pi }_1{\delta }_z{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,0,m\right)\frac{x^m{y}^{m-1}}{\left(m\right)!\left(m-1\right)!}\\ +{\pi }_1{\delta }_y{\delta }_z\end{array}$ (3)

Proof of Lemma 3.

Recall that ${ℒ}^{-1}\left[\frac{1}{u^n}\right]\left(x\right)=\delta \left(x\right)$ if $n=0$ and ${ℒ}^{-1}\left[\frac{1}{u^n}\right]\left(x\right)=\frac{x^{n-1}}{\left(n-1\right)!}$ if $n$ is a positive integer where the latter identity may be obtained by repeated integration by parts. Splitting

${\pi }_{1}vw{\displaystyle \sum _{m=0}^{\infty }{\displaystyle \sum _{l=0}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)u^{-m+l-1}{v}^{-m+k-1}{w}^{-m+j-1}$ (4)

into cases where the exponent(s) of $v,w$ are zero or non-zero, we may ascertain that (4) is equivalent to

$\begin{array}{l}\frac{{\pi }_1}{u}+{\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{l=1}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)u^{-m+l-1}{v}^{-m+k}{w}^{-m+j}\\ +{\pi }_1{\displaystyle \sum _{m=2}^{\infty }{\displaystyle \sum _{k=1}^{m-1}{\displaystyle \sum _{j=0}^{m-k}b}}}\left(m,0,k,j\right)u^{-m-1}{v}^{-m+k}{w}^{-m+j}\\ +{\pi }_1{\displaystyle \sum _{m=1}^{\infty }{\displaystyle \sum _{j=0}^{m-1}b}}\left(m,0,0,j\right)u^{-m-1}{v}^{-m}{w}^{-m+j}\\ +{\pi }_1{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,m,0\right)u^{-m-1}{w}^{-m}\\ +{\pi }_1{\displaystyle \sum _{m=1}^{\infty }b}\left(m,0,0,m\right)u^{-m-1}{v}^{-m}\end{array}$ (5)

Taking the inverse Laplace transform of (5) gives (3) as desired.

Proof of Theorem 1.

Let the joint probability density function of $X\left(t\right),Y\left(t\right),Z\left(t\right)$ be $f\left(x,y,t\right)$ and denote the bivariate moment generating function of $f\left(x,y,t\right)$ by $E\left[{\text{e}}^{-{\theta }_{1}x-{\theta }_{2}y}\right]$ . Then taking the Laplace transform of $E\left[{\text{e}}^{-{\theta }_{1}x-{\theta }_{2}y}\right]$ and applying Lemma 2 for $n=3$ and $k_i=1$ , we obtain

$\begin{array}{l}ℒ\left\{f\left(x,y,t\right)\right\}\left({\theta }_1,{\theta }_2,\varphi \right)\\ ={\displaystyle {\int }_0^{\infty }{\displaystyle {\int }_0^{\infty }{\displaystyle {\int }_0^{\infty }f}}}\left(x,y,t\right){\text{e}}^{-{\theta }_{1}x-{\theta }_{2}y-\varphi t}\text{d}y\text{d}x\text{d}t\\ ={\displaystyle {\int }_0^{\infty }E\left[{\text{e}}^{-{\theta }_{1}x-{\theta }_{2}y}\right]{\text{e}}^{-\varphi t}\text{d}t}\\ ={\displaystyle {\int }_0^{\infty }{\Pi }_3{\text{e}}^{t\left(R-H\right)}{\text{e}}^{-\varphi t}{1}_3\text{d}t}\\ =-{\Pi }_3{\left[R-\varphi I-H\right]}^{-1}{1}_3\\ =\frac{{\pi }_{1}vw+{\pi }_{2}uw+{\pi }_{3}uv+c_{1}u+c_{2}v+c_{3}w+c_4}{uvw-u{\lambda }_{23}{\lambda }_{32}-v{\lambda }_{13}{\lambda }_{31}-w{\lambda }_{12}{\lambda }_{21}-{\lambda }_{12}{\lambda }_{23}{\lambda }_{31}-{\lambda }_{13}{\lambda }_{21}{\lambda }_{32}}\end{array}$ (6)

where

$-\left[R-\varphi I-H\right]=\left[\begin{array}{ccc}u& -{\lambda }_{12}& -{\lambda }_{13}\\ -{\lambda }_{21}& v& -{\lambda }_{23}\\ -{\lambda }_{31}& -{\lambda }_{32}& w\end{array}\right],{\Pi }_3=\left[{\pi }_1,{\pi }_2,{\pi }_3\right],1_3=\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]$

$u={\theta }_1+\varphi +{\lambda }_{12}+\text{}{\lambda }_{13},v={\theta }_2+\varphi +{\lambda }_{21}+{\lambda }_{23},w=\varphi +{\lambda }_{31}+{\lambda }_{32}$

Denoting the numerator of (6) as $N$ and factoring ${\left(uvw\right)}^{-1}$ , (6) may be written as

${\left[\frac{N}{1-\frac{u{\lambda }_{23}{\lambda }_{32}+v{\lambda }_{13}{\lambda }_{31}+w{\lambda }_{12}{\lambda }_{21}+{\lambda }_{12}{\lambda }_{23}{\lambda }_{31}+{\lambda }_{13}{\lambda }_{21}{\lambda }_{32}}{uvw}}\right]}^{-1}{\left(uvw\right)}^{-1}$ (7)

Then for sufficiently large ${\theta }_1,{\theta }_2,\varphi$ we may expand (7) as a geometric series and rewrite it as

$\begin{array}{l}N{\displaystyle \sum _{m=0}^{\infty }{\displaystyle \sum _{l=0}^m{\displaystyle \sum _{k=0}^{m-l}{\displaystyle \sum _{j=0}^{m-l-k}b}}}}\left(m,l,k,j\right)u^{-m+l-1}{v}^{-m+k-1}{w}^{-m+j-1}\\ =ℒ\left\{g\left(x,y,t\right)\right\}\left(u,v,w\right)\\ =ℒ\left\{{\text{e}}^{-x\left({\lambda }_{12}+{\lambda }_{13}-{\lambda }_{31}-{\lambda }_{32}\right)-y\left({\lambda }_{21}+{\lambda }_{23}-{\lambda }_{31}-{\lambda }_{32}\right)-t\left({\lambda }_{31}+{\lambda }_{32}\right)}g\left(x,y,t-y-x\right)\right\}\left({\theta }_1,{\theta }_2,\varphi \right)\end{array}$ (8)

From the equality of (8) and (6), we have

$f\left(x,y,t\right)={\text{e}}^{-x\left({\lambda }_{12}+{\lambda }_{13}-{\lambda }_{31}-{\lambda }_{32}\right)-y\left({\lambda }_{21}+{\lambda }_{23}-{\lambda }_{31}-{\lambda }_{32}\right)-t\left({\lambda }_{31}+{\lambda }_{32}\right)}g\left(x,y,t-y-x\right)$

Finally, by taking the inverse Laplace transform of (8) in a manner akin to the proof of Lemma 3 for all terms gives (1) as desired.

4. Illustrative Example

We present a table with values provided by a Wolfram Mathematica implementation of (1) as well as a simulation of 10⁹ paths in C++. The Mathematica implementation of (1) involves only a summation of index $m$ up to 40 terms instead of the infinite summation of (1), and for the path simulation the proportion of paths satisfying $x_1\le x<x_2$ , $y_1\le y<y_2$ was used to calculate the simulated probabilities. The relevant parameters are as follows.

$R=\left[\begin{array}{ccc}-9& 3& 6\\ 4& -6& 2\\ 1& 3& -4\end{array}\right],{\Pi }_3=\left[1,0,0\right]$

${\displaystyle {\int }_{x_1}^{x_2}{\displaystyle {\int }_{y_1}^{y_2}f}}\left(x,y,5\right)\text{d}y\text{d}x$	$0\le x<1$	$1\le x<2$	$2\le x<3$	$3\le x<4$	$4\le x<5$	$5\le x<6$
$0\le y<1$	0.036051100	0.055836600	0.002708140	0.000005337	0.000000001	0
$1\le y<2$	0.290716000	0.384080000	0.009010030	0.000003378	0	0
$2\le y<3$	0.123652000	0.093462900	0.000392612	0	0	0
$3\le y<4$	0.003429840	0.000652630	0	0	0	0
$4\le y<5$	0.000003147	0	0	0	0	0
$5\le y<6$	0	0	0	0	0	0
Path Simulation	$0\le x<1$	$1\le x<2$	$2\le x<3$	$3\le x<4$	$4\le x<5$	$5\le x<6$
$0\le y<1$	0.036047441	0.055842521	0.002709707	0.000005406	0	0
$1\le y<2$	0.290723689	0.384056728	0.009008985	0.000003493	0	0
$2\le y<3$	0.123652673	0.093470497	0.000393724	0	0	0
$3\le y<4$	0.003433051	0.000649946	0	0	0	0
$4\le y<5$	0.000002139	0	0	0	0	0
$5\le y<6$	0	0	0	0	0	0

Comparison of the simulated values with the exact values in the table above shows good agreement, whereby offering Monte Carlo method as an alternative to exact pdf formula which are expected to be very lengthy for $n$ larger than 3.

Without a closed form expression, considerations must be made when determining an upper bound for index $m$ in (1). An appropriate upper bound for $m$ often depends on $t$ and $R$ where sufficiently large values of $t$ can result in the truncated version of (1) performing poorly. An informal heuristic for gauging the accuracy of the truncated pdf is by confirming that its integral over its domain is sufficiently close to 1. The authors decision to pick the rate matrix and initial distribution above is based off of this observation.

5. Discussions

We would like to note that in principle, one may derive the occupation time probability density function for any finite state Markov chain by utilizing the relevant extension of Darroch’s identity as given by Lemma 2. However, as seen in (1), even for a Three-State Markov chain, the formula is rather lengthy and its derivation is more suited to a computer algebra system. To the best of the authors’ knowledge, (1) is the first explicit example of the occupation time probability density function for a Three-State Markov chain, and is also the first demonstration that Darroch’s identity and Pedler’s derivation may be extended beyond two states. As far as we are aware, Sericola’s approach appears to be the only other method that may be used to derive the occupation time probability density function in a finite state Markov chain; however, we did not find any explicit results in the literature. Concerning Hsia’ result, the probability density function found in Hsia [4] is expressed in terms of random variables with unknown distributions and cannot be directly integrated to calculate numerical values.

On a final note, we would like the reader to be aware that there are several published results for the Two-State occupation time probability density function that contain typographical errors. Among incorrectly typed formulas, the omission of the Dirac delta functional is particularly common and the resulting probability density function fails to integrate to 1 over its domain. Regarding the Two-State occupation time probability density function, Pedler’s formula as written was verified to be correct. As for future work, it would be desirable to have a closed form for various parts of (1) if it exists. Alternatively, new methods for deriving the probability density function may yield more tractable representations of (1).

Acknowledgements

The authors would like to thank the reviewers for their valuable suggestions that considerably improved the presentation of our findings in the revised version of the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Darroch, J.N. and Morris, K.W. (1968) Passage-Time Generating Functions for Continuous-Time Finite Markov Chains. Journal of Applied Probability, 5, 414-426.[CrossRef]
[2]	Pedler, P.J. (1971) Occupation Times for Two State Markov Chains. Journal of Applied Probability, 8, 381-390.[CrossRef]
[3]	Good, I.J. (1961) The Frequency Count of a Markov Chain and the Transition to Continuous Time. The Annals of Mathematical Statistics, 32, 41-48.[CrossRef]
[4]	Hsia, W.S. (1976) The Joint Probability Density Function of the Occupation Time of a Three-State Problem. Journal of Applied Probability, 13, 57-64.[CrossRef]
[5]	Sericola, B. (2000) Occupation Times in Markov Processes. Communications in Statistics. Stochastic Models, 16, 479-510.[CrossRef]
[6]	Fang, C.Y., Liu, Y., Shi, Z.Y. and Chen, C. (2023) Closed-Form Expression of Geometric Brownian Motion with Regime-Switching and Its Applications to European Option Pricing. Symmetry, 15, 575.[CrossRef]
[7]	Guo, X. (1999) Inside Information and Stock Fluctuations. https://www.researchgate.net/publication/2499237_Inside_Information_And_Stock_Fluctuations