^{1}

^{2}

^{*}

In this paper, we will illustrate the use and power of Hidden Markov models in analyzing multivariate data over time. The data used in this study was obtained from the Organization for Economic Co-operation and Development (OECD. Stat database url: https://stats.oecd.org/) and encompasse d monthly data on the employment rate of males and females in Canada and the United States (aged 15 years and over; seasonally adjusted from January 1995 to July 2018) . Two different underlying patterns of trends in employment over the 23 year s observation period were uncovered.

The literature on economic and employment data is extensive and addresses specific issues of interest to government policy, educational, and business concerns. The Organization for Economic Co-operation and Development (OECD) provides data on well-being and economic data from 36 countries (URL: https://stats.oecd.org/) [

Traditionally, time series analysis has been used to analyze employment and related data over time. Many of the time series models (Autoregressive Moving Average (ARMA), Autoregressive Integrated Moving Average (ARIMA), etc.) assume that the mechanism driving the models is constant over time. If the data is observed over a long period of time, it is very likely that there are changing underlying states that generate data from different distinct models. In this instance, the appropriate model of the data is the Hidden Markov Model (HMM). Hidden Markov Models (HMMs) are a widely used collection of statistical models. These models are applicable when studying a process that goes through a sequence of states. These states are unseen (hidden) but what is observed is data from each state. This means that the HMMs have potentially very wide applications across a large variety of fields, including physical and psychological development in human beings as well as the examination of physiological changes in medicine (HMMs have been used to model heart rate variability [

In this paper, we will illustrate the use and power of Hidden Markov Models in analyzing multivariate data over time. The customized data examined here encompasses monthly data on the employment rate of males and females in Canada and the United States (aged 15 years and over; seasonally adjusted from January 1995 to July 2018.) The 23-year period was considered sufficient to uncover any underlying employment trends of interest. For our purposes, and for simplification, we did not include variables, such as age, socio-economic status, part-time vs. full-time employment, geographical location, etc.

A hidden Markov model consists of a sequence of states X_{1}, X_{2}, …, X_{T}, together with a sequence of observations Y_{1}, Y_{2}, …, Y_{T}. The total number of states (possible values of each X_{i}) we will assume is a finite number, m. In this case the states can be represented by the integers 1, 2, ..., m. The states are not observed. The observations Y_{1}, Y_{2}, …, Y_{T} are observed and could be vectors of dimension k. The distribution of Y_{t} depends on the state X_{t}, the state the Markov process is in at time t. In this paper we are assuming that that the distribution of Y_{t} if X_{t} = i (i = 1, 2, … m) is the Multivariate Normal distribution with mean vector μ i and covariance matrix S_{i}.

The parameters of the hidden Markov model are the initial state probabilities,

π i = Pr ( X 1 = i ) i = 1 , 2 , ⋯ , m _{ }

and the transition probability matrix Γ = (γ_{ij}). This is an m x m matrix, with element γ_{ij} being the probability of a transition into state j starting from state i.

i.e.

γ i j = Pr ( X t = j l X t − 1 = i ) ,

where t denotes time. These two choices allow a sequence of states (known also as the Markov chain) X_{1}, X_{2}, …, X_{T} constituting the hidden part of a hidden Markov model.

When the Markov chain is in state i, at time t, it emits an observed signal Y_{t}, which is either a discrete or a continuous random variable (or random vector) with distribution conditional on the current state i.

In the discrete case

Pr [ Y t = y | X t = i ] = p i ( y ; θ i )

where p_{i} is probability mass function with parameters θ_{i}. In the continuous case the conditional density of Y_{t} = y given X_{t} = i is f_{i}(y; θ_{i})

In this paper we will assume that f_{i}(y; θ_{i}) is the Multivariate Normal distribution with mean vector μ i and covariance matrix S_{i}.

There are several statistical questions associated with Hidden Markov models:

1) Given the observed data, what are the estimates of the parameters of model.

a) The initial state probabilities.

b) The state transition probabilities.

c) The parameters of the distribution of the observation vectors Y_{t} when the Hidden Markov chain is in state i. ( μ i and S_{i}.)

2) Given the observed data, and the estimated parameters of model, what are the sequence of unknown states, X_{1}, X_{2}, X_{3}, …, X_{T} that generated the data.

The data used in this study was obtained from the Organization for Economic Co-operation and Development (OECD) and consisted of monthly employment rates on both males and females (aged 15 years and over; seasonally adjusted) over a 23.5 year period (January 1995 to July 2018).

The measure used in this study was monthly rate of employment for each of Canadian and USA males and females as reported in the OECD database.

The initial analysis of the data consisted of graphs of the raw data and changes in the employment rate for all four groups over the time period under consideration. The estimation of the parameters of a Hidden Markov model (HMM) and computation of posterior probabilities of states were subsequently determined. A Hidden Markov statistical model was used to uncover hidden states in the data and the mean vectors and standard deviation for the states was determined. A correlation matrix and principal components were calculated for each state uncovered. The estimation of the parameters of the Hidden Markov model (HMM) and computation of posterior probabilities of states was achieved using the R program, depmixS4.

The data consisting of male and female employment rates over the 23 year period were graphed to give an overall picture.

All graphs indicate periods of both increasing and decreasing trends in employment rates for males and females in both countries. The following Graphs (

These graphs show volatility in the month to month changes in monthly employment rates. Without further analysis it is difficult to assess whether the central values of the monthly changes in employment rates are nonzero (positive or negative) during particular periods of time. We will now apply the use of HMM models to uncover hidden states in trends that are not visually evident in the above graphical data.

The Estimation of the Parameters of a Hidden Markov Model (HMM) and Computation of Posterior Probabilities of StatesThe data that was analyzed was the monthly changes in employment rates of males and females of the two countries over the observed period. Our analysis indicated that two underlying states were uncovered.

From

One can note (

Principal components are a technique for understanding complex correlation structures. The first principal component is a linear combination of the observations that accounts for the maximum percentage of variance in the observations. Subsequent principal components (PCs) are independent of the previous PCs and account for the maximum variance in the observations that has not been accounted by the previous PCs. The first principal component account for all of the variability of monthly employment changes during state 1. The first principal component can be interpreted as increasing or decreasing numbers in the employment changes for all four groups simultaneously. The loadings on the next three principal components result in groupings based on negative or positive sign. The second principal component is a comparison between Canadian and American employment changes when in the first state and account for 23.4% of the total variation in the rate changes. It will be noted that American increase in employment rate occurs with Canadian decrease. The third principal component is a comparison heavily weighted on Canadian males compared with the other three groups. The fourth principal component associated with state 1 is an effect of gender restricted to the USA, notably American females versus American males.

The first four principal components account for approximately the same contribution to variability within monthly changes during state 2. This finding usually occurs when the variables are independent.

In a Hidden Markov model one is interested in determining the underlying hidden state at each point in time suggested by the data. The following graph (

Looking again at

Transition matrix | ||
---|---|---|

to State 1 | to State 2 | |

from State 1 | 0.893 | 0.107 |

from State 2 | 0.025 | 0.975 |

State 1 | State 2 | ||||
---|---|---|---|---|---|

Mean | SD | Mean | SD | ||

Canada | Male | −0.082 | 0.239 | 0.032 | 0.164 |

USA | Male | −0.181 | 0.249 | 0.024 | 0.161 |

Canada | Female | 0 | 0.239 | 0.039 | 0.158 |

USA | Female | −0.118 | 0.237 | 0.027 | 0.173 |

Canada | USA | Canada | USA | ||
---|---|---|---|---|---|

Male | Male | Female | Female | ||

Canada | Male | 1.000 | 0.353 | 0.246 | 0.301 |

USA | Male | 0.353 | 1.000 | 0.135 | 0.526 |

Canada | Female | 0.246 | 0.135 | 1.000 | 0.177 |

USA | Female | 0.301 | 0.526 | 0.177 | 1.000 |

Principal Components | |||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | ||

Canada | Male | −0.499 | −0.204 | 0.828 | 0.156 |

USA | Male | −0.571 | 0.366 | −0.117 | −0.726 |

Canada | Female | −0.331 | −0.853 | −0.389 | −0.107 |

USA | Female | −0.562 | 0.311 | −0.387 | 0.661 |

Eigenvalue | 1.905 | 0.936 | 0.694 | 0.465 | |

% | 47.6% | 23.4% | 17.3% | 11.6% | |

cum % | 47.6% | 71.0% | 88.4% | 100.0% |

Canada | USA | Canada | USA | ||
---|---|---|---|---|---|

Male | Male | Female | Female | ||

Canada | Male | 1.000 | 0.075 | 0.115 | 0.070 |

USA | Male | 0.075 | 1.000 | 0.039 | −0.072 |

Canada | Female | 0.115 | 0.039 | 1.000 | −0.073 |

USA | Female | 0.070 | −0.072 | −0.073 | 1.000 |

Principal Components | |||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | ||

Canada | Male | 0.549 | 0.569 | −0.054 | 0.610 |

USA | Male | 0.506 | −0.206 | −0.770 | −0.331 |

Canada | Female | 0.614 | −0.007 | 0.617 | −0.492 |

USA | Female | −0.256 | 0.796 | −0.155 | −0.526 |

Eigenvalue | 1.166 | 1.069 | 0.959 | 0.805 | |

% | 29.1% | 26.7% | 24.0% | 20.1% | |

cum % | 29.1% | 55.9% | 79.9% | 100.0% |

This example utilizing employment data illustrates the usefulness of the Hidden Markov Model in studying a multiple dimensional process that is changing over time. From this analysis, we see that over a 23-year period (1995-2018), the trends in Canadian and United States employment rates do not follow a single pattern (or single state). The HMM analysis has shown at least two underlying patterns in employment trends. This would be missed by traditional time series analysis which assumes a common state over time. In the first state (shorter lasting), the number of employees in both Canada and the United States is decreasing (with the exception of Canadian women). In the second state, longer in duration, there are small monthly increases in all four groups. In employment data, there are states of high and low volatility. The HMM model provides more precise measurements of how long employment data stay in uncovered states. The conditions that generate observations over time do not stay constant for long periods of time. The underlying conditions (states) change if observed for long periods of time. The HMM takes this into account.

In our analysis, we assumed independence between monthly observation vectors while the process was in a constant state. This assumption could be dropped and assume autocorrelation within states. In this latter case, one would apply an Autoregressive Hidden Markov analysis (e.g. see [

A similar analysis could be conducted over a variety of fields where data is collected over time. The important requirement is not the absolute length of time (23 years in this example) but the frequency of observations and frequency of state changes. For example, the classical example of the use of Hidden Markov models was in speech recognition (Rabiner, 1989) where the length of data sequences was a few seconds. With the increase in the ability to collect longitudinal health and social data, these provide fruitful areas for the application of these techniques.

The authors declare no conflicts of interest regarding the publication of this paper.

Laverty, W.H. and Kelly, I.W. (2018) An Examination of Male and Female Monthly Employment Rates over Time in Canada and the United States Using Hidden Markov Probability Models. Open Journal of Statistics, 8, 837-845. https://doi.org/10.4236/ojs.2018.85055