General Variance Covariance Structures in Two-Way Random Effects Models

This paper examines general variance-covariance structures for the specific effects and the overall error term in a two- way random effects (RE) model. So far panel data literature has only considered these general structures in a one-way model and followed the approach of a Cholesky-type transformation to bring the model back to a “classical” one-way RE case. In this note, we first show that in a two-way setting it is impossible to find a Cholesky-type transformation when the error components have a general variance-covariance structure (which includes autocorrelation). Then we propose solutions for this general case using the spectral decomposition of the variance components and give a general transformation leading to a block-diagonal structure which can be easily handled. The results are obtained under some general conditions on the matrices involved which are satisfied by most commonly used structures. Thus our results provide a general framework for introducing new variance-covariance structures in a panel data model. We compare our results with [1] and [2] highlighting similarities and differences.


Introduction
Panel data models are often characterised by a three component error structure consisting of an individual (time invariant) specific effect, a time specific (individual invariant) effect and an overall idiosyncratic error term varying in both the individual and time dimensions. This leads to a corresponding three component variance covariance structure for the combined error term of the model. Due to the potentially large dimension of this variance covariance matrix, its inverse is usually calculated using matrix decomposition results. In a seminal paper [3] derived the spectral decomposition of the variance-covariance matrix of a two-way (and a one-way) random effects (error component-EC) model when all the components are assumed to be i.i.d. and independent two by two.
Several works have extended Nerlove's result when non-i.i.d. structures are assumed for the error terms, namely, an MA(1) for the overall error term in [4], an AR(1) structure for the overall error term in [5], an ARMA structure for the overall error term in [6][7][8]. All these studies consider a one-way EC i.e. only individual effect in addition to the overall error term, and apply a first-stage Cholesky-type transformation in order to get back to a "classical" one-way EC setting in which both the individual effect and the idiosyncratic error are i.i.d. after the transformation. Once we get back to the "classical" setting, we can apply Nerlove's spectral decomposition for implementing the GLS procedure. Thus, all the studies so far have employed a combination of a first stage Cholesky and a second stage spectral decomposition.
In this note, we first show that the Cholesky-spectral combination approach adopted by all the above-mentioned articles is not possible in a two-way setting and there exists no transformation that will get us back to a "classical" EC structure. Then we propose a solution to the problem through a different first stage transformation based on a new spectral decomposition result which we present and prove. This new result is derived for two way EC models with general variance-covariance structures for all the error components. Our first stage transformation does not yield a classical two-way EC but a one-way EC with heteroscedastic errors and we show that the spectral decomposition and hence the determinant of the variance covariance matrix of the transformed errors is easy to obtain, thus making GLS-and Maximum Likelihood-type procedures operational.
A recent article [1] provides a solution for the inverse and determinant of some general variance covariance structures in panel data models including a general one-way EC setting with a variance covariance matrix of the form 2 where is a A matrix of full column rank, is a  matrix of full column rank, A 2 and B 2 are positive definite matrices of order and respectively. They showed that a two-way EC of the form , where a is a T-vector, b is a N-vector, A is , and B is , is a special case of the above general one-way EC. However, the authors add that if one were to generalise the traditional three component structure, namely 3 , where all the first matrices of the Kronecker products are of order  and the second ones , then "the inverse is not easily computable in general". The above general Kronecker structure is precisely what we consider in our note and we propose a general two-stage transformation for calculating its inverse and determinant under some conditions 1 Thus our result represents a natural follow up to the note by [1]. Our note partially validates their conjecture as our first stage spectral transformation yields a two component heteroscedastic structure and we do bring the model to a two component variance structure at an intermediate stage. However, we say only "partially" as we cannot rewrite our original variance matrix as a two component variance structure but apply a transformation to convert a general three component variance structure into a two component one.

N N 
Another more recent study [2] presents a general approach for a two-way EC under double autocorrelation in both the time effect and the idiosyncratic error. They propose a transformation based on a mixture of Cholesky and spectral decompositions as a first stage (what we call a "hybrid" transformation in this note), and a spectral decomposition as a second stage. In our note, we use our methodology to extend their approach to more general structures at the cross-sectional level. We derive the spectral decomposition as well as the determinant of the resulting variance-covariance matrix whereas [2] only derives the inverse of the transformed variance-covariance matrix.
The paper is organised as follows. Section 2 shows why Cholesky-type transformations do not work in the two-way EC model. In Section 3 we derive the new "spectral-spectral" combination result for general structures of variance-covariance matrices. The new transformation is obtained under some conditions that are satisfied for most of the structures usually encountered in a panel data setting. Section 4 takes up some of these commonly used structures and describes how a solution can be found for these structures using our new decomposition result. Section 5 presents a new transformation along the lines of [2]. Finally, we conclude by pointing out some interesting aspects of our approach that may be worth investigating further.

On the Impossibility of Cholesky Decomposition in a Two-Way Error Component Context
Consider the following panel data model: , 1, , , 1, ,  denoting the individual specific random effect, t  denoting the time specific random effect, it  the overall idiosyncratic error term, and where β is a 1 k  vector of coefficients including the intercept, i and t denote the individual and the time period respectively, it is a x 1 k  vector of observations on k strictly exogenous explanatory variables. it  is assumed to follow a stationary process t  independent over , and parameterised by a vector . Writing  Ω θ is any non-iid variance covariance structure and the disturbance components are independent two by two. Under these assumptions, the variance-covariance matrix of the linear model can be expressed as follows: In the case of a one-way EC model, the structure typically proposed in the literature for is an autocorrelated structure and a Cholesky decomposition is used to get back to a "classical" one-way EC setting. In the following lemma, we establish that in a two-way EC model it is impossible to find a Cholesky decomposition that diagonalises

Spectral Decompositions, an Alternative
This section shows that two-stage spectral decompositions can be applied to solve the problem of an analytical inverse of V for the general variance-covariance structure (2) presented in the previous section. Before deriving the main result in the form of a theorem, we illustrate our idea in the case of a two-way EC model with autocorrelation in the overall error term, a case that may be commonly encountered in practice (for which there is no explicit solution so far). Assume that we are dealing with a panel data model with the following variance-covariance matrix (EC with MA structure for the idiosyncratic error): As the inverse of the above variance-covariance matrix is rather difficult to compute, if we want to apply a GLS or Maximum Likelihood procedure we need to find a transformation which will yield a variance-covariance matrix with a tractable inverse. In Section 2, we proved that it was not possible to get to the classical structure through the Cholesky decomposition. Nor is it possible to rewrite the model as a one-way EC structure. Here we show that by using the spectral decomposition, we can provide an explicit transformation that allows us to get to a model with a variance-covariance matrix that is easy to handle.
First we note that [9] has given the orthogonal matrix such that , where , , 2 π 2 π π sin ,sin , ,sin 1 Next we derive another result (Lemma 2 below) which gives the diagonalisation of a unit matrix and shows that the same orthogonal matrix also diagonalises an equicorrelation matrix as well. This can be particularly useful in panel data models with both individual and time specific effects. In fact, one of the main problems for getting the spectral decomposition of the full variance-covariance matrix in the presence of a time effect is the cross-sectional dependence induced by the latter. Indeed, in the presence of a time effect we lose the block-diagonal structure which is found in one-way models. This lemma is useful as it allows us to obtain a transformation leading to a block-diagonal structure.
Then the same matrix respectively. Now combining [9]'s result and Lemma 2, we can give a transformation, 1 which when applied to (3) will lead to a block-diagonal structure whose spectral decomposition is easily obtained (see Theorem 1 and Special Case 1). Thus GLS and ML methods become much easier to operationalise in presence of complex variance covariance structures.
The method presented above fits in a more general approach that provides a way out for obtaining the inverse of any general variance-covariance structure for the specific effects as well as the idiosyncratic error in a two-way EC setting. We derive the solution under certain assumptions which may seem restrictive at first sight but which we show to be satisfied by many general structures frequently found in the literature. In fact we also show that this theorem enables us to introduce some new and possibly relevant structures in panel data models.
Let us now consider general processes for ,  2) The spectral decomposition of the transformed variance-covariance matrix is given by: 3) The determinant of the transformed variance-covariance matrix can be written as: 2) Our results are derived under the double commutativity constraint (5). Though this condition looks rather restrictive, it turns out to be quite general in the panel data framework. In fact, not only all the structures studied so far in the panel econometric literature satisfy this condition but also there are other possibly interesting structures which have not yet been considered that can be included in this setting. One such structure is equicorre-effects in certain empirical contexts. It can even be modified to reflect block dependence among individuals (correlation within clusters).
3) We show different ways of taking cross-sectional dependence into account, i.e., through  Ω  Ω . be and/or The presence of the time effect is already a source of cross-sectional dependence in our mod nd it can further generalised to have a fuller variance-covariance structure through el a  Ω . This feature can be linked to the more recent strand of literature that deals with crosssectional dependence through factor analysis. 4) Although our result is general, it may present some operational difficulties. Our transformation matrix requires the knowledge of eigenvectors and eigenvalues of the different matrices involved and it may be cumbersome to actually determine these for some structures. Unlike the MA case that we saw earlier, in the case of an AR structure for  or  , there are no general analytical expressions available for the eigenvectors of the resulting variance co ariance matrix for any dimension T. These expressions crucially depend on the size of T. Plus there is no recursive way of finding the roots of characteristic polynomials of size T + 1 given those of size T v and one has to calculate them separately for each T. These are practical obstacles to be overcome before implementing our transformation. In spite of this, we believe that our transformation is highly useful as it definitely reduces the size of the problem in all circumstances i.e. instead of finding the eigenvalues/eigenvectors of a matrix of size NT (i.e. of the full variance-covariance matrix of the model), one only needs to find them for a matrix of size T (i.e. of the AR structure over just the time dimension). In the worst case scenario, one can compute these eigenvalues and eigenvectors numerically in the first stage.

Some Special Cases
In this section we will examine some special cases that way models and show how them to the two-way case are commonly used for one our result enables to extend and obtain the spectral decomposition of the variance covariance matrix of the model in a rather straightforward manner. In addition, we also introduce some new structures that have not been considered so far in this context. Then, the variance-covariance of the whole model writes: Using Theorem 1, the transformation to be applied to get a block diagonal structure is given by 1 As in case 1, the sp ed variance-cova puted using the resul sformed variance-covariance matrix is: and the tran 4 , , 1 1 , , , , , Theorem 1, as in the a 5) Equicorrelated or e have them for T = 5 for example, we do not present them here as the expressions of the eigenvectors are indeed long. One can also calculate them numerically. Once the first stage decomposition is obtained (perhaps numerically for the third AR component), the procedure in Theorem 1 can be implemented as in the above cases.

Extension to "Hybrid" Transformations
In spite of th a mixture of Cholesky and spectral decompositions (what we call "hybrid" transformations) in the first stage and a spectral decomposition in the second stage. The following theorem provides a new transformation in this general setting that allows us to obtain the spectral decomposition of the resulting variance-covariance matrix.
Theorem 2. Given a general variance-covariance structure if verified in many panel data models of practical relevanc one could argue that it is stringent and may exclude som J Ω Ω (17) potentially important situations. For instance, assume that the researcher expects to have an autocorrelated process not only in the idiosyncratic error but also in the time effect. In general, the variance-covariance matrices of the two autocorrelated processes do not commute. [2] proposes a 3-stage transformation that circumvents the double autocorrelation problem. They successively apply the Cholesky and the spectral transformations to obtain a simpler structure for which they are able to give the inverse. Thus their first two stages are equivalent to our first stage in the sense that after two transformations they provide a way to calculate the inverse of the resulting variance-covariance matrix. However, they do not provide the spectral decomposition of the Cholesky transformed variance-covariance matrix in the first stage as it cannot be explicitly derived.
In this section we examine these more general cases and give appropriate transformations based on our previous results. These transform then the following results hold: 1) There exists a transformation matrix of the form 2) The spectral decomposition of the transformed variance-covariance matrix is given by: and , , c  

Q
, and Cholesky,  C , decompositions in the first stage in order to arrive at a tractable variance-covariance matrix.
3) These transformations ore general than those of are m Theorem 1 in the sense that they can handle more complex structures in  Ω and  Ω , in particular acorrelation of the AR type. However, the problem of finding such transformations becomes even more difficult than in Sectio uto n 3.
we have an idea of th Indeed, even if e Cholesky decomposition of the variance-covariance matrix of a particular stochastic process, say  Ω , such knowledge is not, in gen l, info tive for the spectral decomposition of the variance-covariance matrix of another stochastic process, era rma  Ω , nor for a linear transformation of the latter,     C C Ω . Yet, we require all these decompositions to implement the procedure.
4) The results of this section are only theoret as we cannot provide any explicit transformation, especially in the first stage, as mentioned above. However, our earlier Theorem 1 provides an expl it solution under the commutativity constraint. 5) We still have the commutativity constra ical int between ic  Ω and N J in our case that extends [2]'s model to cover general variance covariance structures at the crosssectional level 5 . We conjecture that in the two-way EC this is the maximum generality that one can afford in order to get back to a heteroscedastic trans one-way case a th ype formations and propose spectral decomposition, -sectional dependence neral structures at the cross-sectional level using ou [

Concluding Remarks
In this paper, we examine general variance-covariance structures for the specific effects and the overall error term in a two-way error component model. We show the limitations of Cholesky-t a different approach, based on for dealing with time and cross . Our transformation can be applied to any general variance-covariance setting, under the commutativity constraint, and we show how this transformation works in many interesting special cases. We also connect our result to [1] and their conjecture for a two-way EC which seems to be verified albeit only after an initial transformation.
As our transformation is based on eigenvalues and eigenvectors of the variance-covariance matrix of the combined disturbance term, we believe that it is strongly linked to the more recent approach of taking cross-sectional dependence into account by means of factor mod-els. We conjecture that our approach is equivalent to the factor approach under some assumptions and we hope to investigate the link between these two approaches in the future.
Finally, we show how the result derived in [2] for the double autocorrelation structure can be extended to cover more ge r method. We provide the spectral decomposition as well as the determinant of the variance-covariance matrix of the transformed model.
In order to get back to a classical EC structure we would need two conditions: 1)   6 One can show that the only possibility that a Cholesky type transformation will allow us to get back to a classical EC will be when λ t and ε it follow the same stochastic process. As this hypothesis is very stringent, one needs to find new transformations in the general case.
where . Hence, q  