Small Sample Estimation in Dynamic Panel Data Models: A Simulation Study

We used simulated data to investigate both the small and large sample properties of the within-groups (WG) estimator and the first difference generalized method of moments (FD-GMM) estimator of a dynamic panel data (DPD) model. The magnitude of WG and FD-GMM estimates are almost the same for square panels. WG estimator performs best for long panels such as those with time dimension as large as 50. The advantage of FD-GMM estimator however, is observed on panels that are long and wide, say with time dimension at least 25 and cross-section dimension size of at least 30. For small-sized panels, the two methods failed since their optimality was established in the context of asymptotic theory. We developed parametric bootstrap versions of WG and FD-GMM estimators. Simulation study indicates the advantages of the bootstrap methods under small sample cases on the assumption that variances of the individual effects and the disturbances are of similar magnitude. The boostrapped WG and FD-GMM estimators are optimal for small samples.


Introduction
Panel data combine cross-sectional and time series information.Since the temporal dependencies for each unit could vary significantly, a dynamic parameter is desirable to relax the parametric constraints into the model.Dynamic panel data (DPD) model postulates the lagged dependent variable as an explanatory variable.Just like in univariate time series analysis, modeling the dependency of the time series on its past value(s) gives valuable insights on the temporal behavior of the series.[1] noted that many economic relationships are dynamic in nature and the panel data allow the researcher to better understand the dynamics of structural adjustment exhibited by the data.
A good number of dynamic panel data estimators have been proposed and thoroughly characterized in the literature.The within-group (WG) estimator, among the early estimation method for DPD, provides consistent estimate for static models.In DPD models, [2] showed that the WG estimator of the coefficient of the lagged dependent variable parameter is downward biased and the bias only disappears as the number of time units grows larger.Thus, the WG estimator is known to be biased whenever the time-dimension is fixed, even if the cross-section dimension is large.

T N
The inconsistency of the WG estimator leads to the development of DPD coefficient parameter estimators that are consistent for large and fixed or large , e.g., the use of instrumental variables (IV).For the IV estimators, [3] used either the dependent variable lagged two periods or its first-differences as instruments.Even the development of the generalized method of moments (GMM) estimators for DPD coefficient parameters is based on the IV approach.[4] proposed GMM estimator that uses all available lags at each period as instruments for the equations in first differences, this is now known as the first-difference generalized method of moments estimator (FD-GMM).[5] proposed the level GMM estimator which is based on the level of the model and uses lagged difference variables as instruments.[6] further proposed the now called system GMM estimator which uses both the lags of the level and first difference as instruments.

N T
The DPD model estimators have exhibited good asymptotic properties, see for example [2], [7], and [8].Some work investigated the small sample properties of the said estimators, e.g., [9] and [10].There are numer-T ous studies on the properties of dynamic panel data model but are mostly focusing on data sets with large cross-section and small time dimensions.Other studies highlight datasets with sizeable cross-section dimensions and moderately-sized time dimensions.
We used intensive simulations to investigate both the small and large sample properties of two of the simplest and oldest DPD estimators, the within-groups and first-difference generalized method of moments estimators of the AR(1) DPD model.We also propose the use of parametric bootstrap procedure in the WG and FD-GMM for the boundary scenario, i.e., when asymptotic optimality of WG and FD-GMM fail.
As [11] pointed out, the application of bootstrap methodology in panel data analysis is currently in its embryonic stage.The bootstrap estimators proposed in this study can answer the possible limitations of the estimators by [8].Over a short period, it is common for processes over time to be easily affected by random shocks.Thus, if long period data are used, it is very likely that structural change will manifest and the modeler will either incorporate the change into the model (more complicated), or analyze the series by shorter periods, leading to small sample data where the proposed estimator is applicable.

Dynamic Panel Data Model
Suppose the dynamic behavior of a time series for unit i (y it ) is characterized by the presence of a lagged dependent variable among the regressors, i.e.

O 
Consider the following AR (1) where it is the dependent variable, y is the unobserved heterogeneity or individual effect which has mean 0 and variance  and it unobserved disturbance with mean 0 and variance To facilitate the computations of estimators of model 2, let T  vector of ones).Equation (2) can be written as [8] considered several estimators of  , e.g., the within group/covariance (WG) estimator (also called fixed effects (FE) estimator), covariance (CV) estimator (or least squares dummy variable (LSDV) estimator), and the first-difference generalized method of moments (FD-GMM) estimator (one-step level GMM estimator proposed by [4]).In the computation of WG  and FD-GMM  further notations are used: ,..., , where , T is called the WG operator of order T. WG estimator may also be written in terms of the forward orthogonal deviations operator Q A , an upper triangular-like matrix, with dimension and with the following characteristics: , 1 T 

and
. Let and , then WG estimator is given by   [8] further analyzed an asymptotically efficient FD-GMM estimator given by where .A computationally useful alternative expression for x M x (7) where t and t are the vectors whose i th elements are and it , respectively, and t is the matrix whose i th row is .As pointed by [8], is nonsingular when , but the projections involved remain well defined in any case.Without loss of generality, the condition 1 was maintained because the FD-GMM estimator is motivated in a situation where T is smaller than N and it is straightforward to extend the results in their paper to allow for any combination of values of T and N by considering a generalized formulation of 7 using , where , the asymptotic variance of FD-GMM estimator is the same as the WG estimator and they have similar expression for their (negative) asymptotic biases in which for FD-GMM has order    , the asymptotic bias of GMM is smaller than the bias of WG.When , the two asymptotic biases are equal and when , the asymptotic bias in the WG estimator disappears.

The Bootstrap Method
The bootstrap is a useful tool for estimation in finite samples.Bootstrap procedure entails the estimation of parameters in a model through resampling with a large number of replications, [15].[16] developed the idea of bootstrap procedure known as a nonparametric method of resampling with replacement and it stems from older resampling methods such as the jackknife method and delta method.Originally, the bootstrap requires independent observations, i.e., consisting of observations 1 x , …, n x , a random sample from the true distribution   F x generating the data.The data gener- , a discrete distribution that assigns equal probability to the observations of the observed sample, hence, . The b th bootstrap sample is a vector  On the other hand, the block bootstrap resamples from overlapping blocks of consecutive observations to generate the bootstrap replicates, see [17] and [15] for a more comprehensive discussion of bootstrap methods for time series models.
To define the bootstrap for dynamic panel data models, suppose is defined as measurements from different cross-section units of the population over different time periods, so that the data can be represented by b is generated by doing the AR-sieve bootstrap procedure for panel data.[11] identified five bootstrap methodologies for panel data namely: i.i.d.bootstrap, individual bootstrap, temporal bootstrap, block bootstrap and double resampling bootstrap.The i.i.d bootstrap refers to the bootstrap procedure defined by [16].Each of the  9).The resampling procedure for the block bootstrap is in the temporal dimension, so that the data matrix of the form ( 8) is used.The difference between the block bootstrap and the temporal bootstrap is on the sampling of blocks of columns of PD y in ( 8) instead of single column/period in the temporal bootstrap case.Let T Kl  , where is the length of a block and thus, there are l K non-overlapping blocks.Block bootstrap resampling entails the construction of b such as (9) with columns obtained by resampling with replacement the non-overlapping blocks columns of

Bootstrap Procedure for Dynamic Panel Data Estimators
While the literature clearly illustrates the asymptotic optimality of WG and FD-GMM estimators, there are doubts on their performance under small samples.Many panel data are usually formed from small samples of time points and/or panel units because of the structural change or random shocks that may occur in bigger/larger datasets.For small samples, we propose to use parametric bootstrap on WG and FD-GMM in mitigating the bias and inconsistency that these estimators are known to exhibit for small samples.We consider the AR(1) dynamic panel data model: , where  is the parameter, i  is the individual effect with mean zero and variance 2   , and disturbances it  with mean zero and variance 2   .The bootstrap proce- dure below uses AR Sieve in replication, steps follow: generated from the model in 10, we have time-series data N   For each cross-section unit , we assume an AR(1) model with slope Using the method of least squares we obtain the estimators 1, 2, t ,  T ˆi  and ˆi  for the parameters.
Step 2: Compute  , the average of all ˆi  's over all cross-section units, i.e., Step 3: For a fixed cross-section unit , the predicted values Step 4: Reconstruct the panel data using Step 5: Do step 4 B times, taking note that the used set of it    should not be used again in subsequent reconstruction of the panel data.There will result to panel data Step 6: Compute WG and FD-GMM estimators using Equations ( 5) and ( 7) respectively, for each of the panel data sets.

B
Step 7: Resample the WG and FD-GMM estimates in step 6.

B
When sample size is small, there is a tendency for estimators based on asymptotic optimality to become erratic.The AR sieve is used to reconstruct as many time series as possible that capture the same structure as the original data.Resampling from each of the recreated data and computation of WG and FD-GMM for each resamples can alleviate instability caused by small samples, inheriting the optimal properties of the bootstrap methods.

Simulation Study
In the simulation study, we used AR (1) with individual random effects model given in Equation (10), i.e., We used a Monte Carlo design that aims to examine both the asymptotic and finite (small) sample properties of the two estimators of the parameter α.Where the asymptotic properties are examined, the cross-section dimension go as large as 500 (see [18]) or as small as 50 (see [8]).We consider 50 N  corresponding to the large cross-section dimension scenario.On the other hand, 50 T  is the largest time dimension used in studies about asymptotic properties (see [8]).Some studies use smaller T to show the asymptotic properties of estimators, especially the GMM type, see [8].We assume large time dimension with .When smaller is used such as, (see [10], [19]), even is as large as 100, it still exhibit small sample properties of the estimator, specifically the GMM estimator.We consider as small time dimensions cases with ; , for details, see [9].

T  30 T 
There are few studies where the time series and crosssection dimensions are both small such, e.g., , T = 10, 20, see [7].Small cross-section dimensions considered are , and moderate cross-section dimensions are 30 N  and .We also consider as moderate time dimension cases where 40 N  10 T  and 25 T  .Therefore, the values for and can capture the settings for both short and wide panel, typical of a micro-panel and long and narrow panel, which is a common set-up for macro-panels.

T N
In panel data, the observations in a particular crosssection unit comprise a time series.Since, we employ a dynamic panel data model, the AR(1) coefficient parameter  , can be viewed as the common slope parameter for the time series in an panel data.Thus, given a time series with T observations, our choice of the values of the AR(1) coefficient parameter for each cross-section unit , using Equation (10), the initial , where .
  0 0 . This will give rise to a data matrix of the form 11,   The analysis for the asymptotic and finite sample properties of the WG and FD-GMM estimators was done using 100 replications, in this case, 100 panel data sets PD y for each of the 625 designs/parameter combinations.The WG and FD-GMM estimates for a total of 62,500 data sets using Equations 5 and 7 respectively.The mean, median, quartiles of WG and FD-GMM estimate for each of the 625 sets containing 100 replicates of data are computed.
We compare the performance of the two estimators ˆWG  and ˆFD GMM   using the sample medians and interquartile ranges.Note that the two estimators are both downward biased, so that comparisons will be more meaningful if resistant measures are used to assess the bias and efficiency.Hence, in assessing the finite sample properties of the two estimators, the median bias and median percent bias are used.On the other hand, efficiency is examined by looking at the dispersion using a more resistant measure like the interquartile range as compared to the standard deviation.
If we denote the WG and FD-GMM estimates from the th replicate by r    and the corresponding interquartile range for the FD-GMM estimator is equal to .
[8] computed the asymptotic approximations to the bias given by   for WG and FD-GMM estimators, respectively.

Results and Discussion
We report the Monte Carlo simulations on the WG and FD-GMM estimators for various combinations of values of and T , and relatively wide range of values for N  and 2 2       .The main focus of the analysis is on the bias of the WG and FD-GMM estimators as the number of cross-section dimension and the number of time dimension changes.The change in bias as the true value of the coefficient parameter N T  varies is also shown.The effect of the variance ratio between the individual effect and the random disturbance on the bias of the two estimators is explored.

Effect of the Sample Size on the Bias
The marginal effect of varying the cross-section dimension and the time dimension T are presented separately.The joint effect of and T is also presented as [8] emphasized that the asymptotic bias of the DPD estimators depend on the relative rates of increase of and T .
N N N In theory, the cross-section dimension has no effect on the bias of the WG estimator.This is confirmed N in the simulation exercise, where the bias of WG estimator is relatively constant as varies, given that is fixed.On the other hand, the theory is that the FD-GMM estimator has a bias of order 1 N , that is, the bias decreases as becomes large.This pattern is not perfectly observed in the simulation.For instance, when the variance ratio 1.25 The time dimension affects the bias of the WG estimator.The WG estimator has bias of order T 1 T , that is, as becomes larger, the WG bias becomes smaller.This is also observed in the simulation.As expected, as increases from 3 to 50, the bias reduces tremendously within acceptable levels as T is nearing 50.This pattern is implied by the increase of the magnitude of the medi-ans as becomes larger.
T T T The FD-GMM estimator is known to be affected by the cross-section dimension , and the order of bias is N 1 N and does not involve T 8] noted that consistency of the FD-GMM estimator requires   . [ 2 log 0 T N  and T  s mean he bias of the FD-GMM estimator does not depend on N alone but also T . he FD-GMM esti ates in the simulation exercise illustrate the decrease in bias as the time dimension increases, most specially for 0.5 where hi s that t m . Also, from and within 4% to 27% of the true value of the coefficient, implying the magnitude of -GMM bias FD de increases.creases as T T A summary of the range of values of bias as percentage of the true parameter value for the WG estimator with varying time dimension is presented in Table 3.For a small cross-section dimension , one could expect a WG estimate with a bias from 27% to 132% for 10 N  10 T  , from 10% to 54% for and from 5% to 28% of the true value of the coefficient when 25  T 50 T  .Even for large cross-section dimension, say 50 N  , the WG estimates have similar range of percent bias as those estimate when 10 N  .This indeed shows that there is notable decrease in the bias of WG estimates as increases whatever the value of .
T N Table 3. Percent bias of ˆWG  .
1.25 It is interesting to note that the relationship een ia betw b s and the sample size represented by the pair (N,T) also takes into account the relative rates of increase of N with respect to T for the WG estimator and T with respect to N for the FD-GMM estimators.When we have a square panel, that is when the sample size is either, ( 10 N  , 10 T  ) or ( 50 N  , 50 T  ), the WG estimates and GMM estimates are almost the same.The similarity of the WG estimates to the GMM estimates is also seen for an almost square panel, such as ( 20 N  , 25 T  ), ( 30 N  , 25 T  ), and N = 40, T = 50.This confirms the theory of [8] that the asymptotic bias of WG and FD-GMM are the same when N T  , we confirmed here to be also true for moderate samples and even small samples.When 50 T  , regardless of the size of N , the value of WG estimates are similar to the value of FD-GMM estimates.This is attributed to the fact that in the previously stated scenario, the time-series dimension is always greater than or equal to the cross-section dimension, in this case the workable formula for FD-GMM whenever t N  is almost identical to the WG formula.Both the WG and FD-GMM estimators are do ward sed, that is, the estimates are smaller than wn bia the true value of the coefficient parameter  .As shown by [2] for WG estimator, the bias decreases as  increases.This is true for both the WG and FD-GMM estimators as illustrated in the study.Also, one may think that the percent bias will increase as we decrease the value of  , since the smaller the value of the denominator the larger the fraction becomes.This is confirmed in the results of simulation, the FD-GMM bias decrease as T increase provided that  is large.
The exact distribution of WG estimator is said to be invariant to bo the variance th of the individual effect 2   and the variance of the random disturbance 2   , while the distribution of the FD-GMM estimator is invariant only to the variance ratio 2 2       , see [8].In the simulation exercise, varying the variance ratio does not show sizeable changes on the bias, when the sample size

 
, N T and the value of the parameter coefficient  are fixed.

WG and FD-GMM Estimators
We compar

Other Asymptotic and Finite Sample
Properties of betw ported in Table 4.The following asymptotic properties: e the median of the estimators to the approximate bias values computed by [8].Percent different een the computed bias and the approximate bias are re (a) when T N  , the asymptotic bias of GMM is smaller than the WG bias, (b) when T N  , the expression for the two asymptotic biases are equal and (c) when 0 N T  , the asymptotic bias in the WG estimator disappears, der y Alvarez and Arellano (2003) still hold for smaller samples considered study.The findings of [7] that the WG is more efficient than GMM ported by the simulation study.Since the bias of WG estimator does not depend on N , the values are similar, that is, the number of WG estimates with percent difference values less than 5% is almost the same for different values of N .On the other hand, percent difference of FD-GMM estimate increases with N .The asymptotic approximation of [8] performs well for 40 N  , since 72.8% of the MC medians of FD-GMM estimates are within % of the approximated value. equal to zero, we see that majority, that is, 20 out of the 25 cases considered have bias of M less than the bias of WG.On the other hand, when 2 0    , half of the cases have bias of FD-GMM smaller than bias WG and the other half have bias of WG less than or equal to bias of FD-GMM.The cases where bias of WG is less than or equal to the bias of FD-GM timates have moderate to large T , but when 10 T  , the bias of WG is smaller and closer to the FD-GMM bias only when FD-GM of M es  is at least 0.5.Specifically, bias of WG is close to bias of FD-GMM for square panels where 0.5 and for 25 T  .We nalyzed moderately-sized cross-section di-mension, i.e., 3 N 0  .This 0% the 125 ca allows for 8 of ses to have T N  and the other 20% are designs where T N  .pect that more percentage of FD-GMM estimates maller bias than WG estimates as compar where the cross-dimension size is small.There are 90 (72%) of the 125 cases where bias of FD-GMM is smaller than the bias of WG.The other 35 (28%) cases have either large time-dimension, that is 50 T


or larger value for the coefficient parameter, which is 0.8

 
. This is intuitively true, since when 50 T  , N T  and the bias of WG is expected to be less than of FD-GMM.For moderately-sized mension, about 73% of the FD-GMM estimates have smaller bias than their WG counterpart and the other 27% have designs where the time dimension is large or the value of the coefficient of parameter is close to one.Some 80% of the 125 cases have designs where T < N also a the bias cross-section di

Cases
Numbe tisfying the inequality

Table 5. Comparison of bias of and FD-GMM estimates.
r of cases with Bias(FD-GMM) < Bias(WG) Set-up not sa    e FD-GMM mates have smaller bias than the WG estimates and when square panels are considered the biases are the same.There are 104 (83.2%) of the 125 cases considered has FD-GMM bias smaller than their WG counterpart.The other 21 (16.8%)cases have designs where the time dimension is large, that is, 50 T  and the coefficient parameter is close to one, that is,   0.8 or 0.9.
Table 6 summarizes comparison of the variability of WG and FD-GMM estimates as easured by the interquartile range (IQR).The WG es ates generall m tim y have less variability than FD-GMM estimates, for the mode-  To be able to assess the benefits from boostrapping the WG and GMM estimators, we identified settings in the mulation scenarios where they yield the largest bias.si We report in Tables 7 and 8 the top 25 of estimates wit the largest bias for both WG and GMM, together with respective design specifications.th Thi ong ite surprising for GMM estimator.When 3 T  , FD-GMM estimator uses only one instrument and thus equivalent to the IV estimator, less appealing than the GMM estimators.Moreover, the FD-GMM estimator when 3 T  , does not show a decrease in bias as N becomes larger.
It is known that the bias of WG estimator increases with the coefficient parameter  , see [2] and [7].s simulation exercise, all the WG and GMM worst esti-mates came from designs with large In thi  , see   ), the bootstrap FD-GMM is better than the conven -GMM for cases where the variance of the individual effect is as large as 1 or 1.25.Also, the bootstrap WG maller than the conventional WG when

Conclusions
In estimating a dynamic panel data (DPD) model using WG and FD estimators cros on N has no effect on the bias of the WG estimator, but the bias of FD-GMM decreases as N increases in some cases.The bias of WG estimator decreases as and T = 1 20).
WG estimates are less variable than FD-GMM estimates based on the interquartile range.The WG estimator is best to use when the time-series dimension is as large as 50 and the cross-section dimension as low as 10 for models here th of the varia f the individual effect to the variance of the random disturbance to be less than one and regardless of the true v e ent para t 50 , 20 N 0, w e ratio nce o alue of th coeffici meter.Also, when he T   and 2 2 1   tions will assure that the percent bias of the WG estimate will not exceed 20%.The FD-GMM estimator has bias not exceeding 20% of the true value of the parameter coefficient for small and moderate samples in terms of N when: (


elements of the observed data matrix PD y is given 1 NT probability in the empirical distribution   F y  .The elements of the b th bootstrap sample b  y in (9) are obtained by resampling with replacement from the empirical distribution   F y  .The rows of PD y in (8) are resampled with replacement in order to determine the b th bootstrap sample b  y of the form in (9) in the individual bootstrap procedure.On the other hand, in the temporal bootstrap procedure, the columns of PD y in (8) are resampled with replacement in order to create the b th bootstrap sample b  y of the form ( matrix PD y , double resampling is a procedure that constructs the b th bootstrap sample b  y by resampling columns and rows of PD y .Two schemes can be chosen, the first is a combination of individual and temporal bootstrap and the second is a combination of individual and block bootstrap.Therefore, as the name of this bootstrap procedure implies, it involves two/double stages.The first stage is to construct an intermediate bootstrap sample b  y by performing individual bootstrap.The second stage uses the intermediate bootstrap sample b  y as a matrix where either temporal bootstrap or block bootstrap is applied to produce the b th bootstrap sample b  y .
ith element of the first column of matrix 11.

5 T
 .If the objective is to examine the finite sample properties of estimators especially the WG estimator, small to moderate sizes for both the cross-section and time dimensions are commonly used, e.g.,

.
The sample median for a particular design is estimator.The interquartile range for WG estimator is , only 10 out of the 25 cases have shown the pattern of decreasing bias of the FD-GMM estimates as the number of cross-section dimension increases.Also, when the variance ratio , only 10 out of the 25 cases have shown the pattern of decrease in bias as increases.A summary of the range of values of bias as percentage of the true parameter value for the FD-GMM estimator focusing on varying the cross-section dimension is presented in

. 2 . 2  2 
Effect of Para 6 meter Values on the Bias y has The first-order DPD model considered in this stud three parameters, but we focused only on the coefficient of the lagged dependent variable ( ).The two other parameters are the variances of the one-way random effects error component, namely the variance of the individual effect and the variance of the random disturbance .Instead of analyzing the effect of 2
. In this way,

Table 1 .
from an almost white noise series, where  The generated    .Table 1 presents the dif- we generate

Table 2 .
For a moderate time dimension

Table 9 .
As el

Table 9 . Monte carlo designs yi ding worst estimates (33 parameter combinations).
 , and time the variance of the individual effect is small but nonzero, i.e., , WG and FD-GMM estimates are similar.WG and FD-GMM estimators are both downward biased, the bias increases with  .However, t The bootstrap estimators for both WG and FD-GMM, labeled as WGb and FD-GMMb respectively, work well for the smallest sample size, i.e.,