Econometric Modeling and Model Falsification

A recent literature on qualitative analysis has shown that its successful application in testing the consistency of the sign patterns of a proposed structure and an estimated reduced form was far less restricted than a previous literature had proposed. A frequent example used in this demonstration was the qualitative analysis of Klein’s Model I. For this, the proposed structural sign pattern was falsified by the sign pattern of the estimated reduced form. As a result, the subsequent application two-stage least squares would always find quantifications of the structure that could not possibly have resulted in the sign pattern of the estimated reduced form. We view this result as a diagnostic calling for further analysis. We show that the Klein model fails standard over identification tests. We make modest amendments to the model that re-solves this problem but find that the resulting estimated reduced form still falsifies the structure, calling for further developmental effort. Our point is that qualitative falsification should be viewed as a diagnostic in developing a model, rather than a criterion for entirely dismissing the model.


Introduction
The issue of the scientific nature of economic theory is surprisingly undeveloped; or, at least, not fully formulated. To set up the issues at stake, we will follow Samuelson [1] and later writers in econometrics, e.g., Berndt [2]. In these terms, the theory concerning some aspect of economic activity, i.e., a model, is expressed by a system of simultaneous equations, For purposes of estimation, it is usual to assume that (1) is (at least locally) linear, so that (2) can be more compactly expressed as, where β, γ and δ are appropriately dimensioned matrices, with δU representing errors embodied in the data. The system (3) is termed the structural form of the model. Estimation then proceeds with respect to the reduced form, where in (4) The arrays β and γ are then recovered from the estimated π using some version of multi-stage least squares. Depending upon issues of identification, there may be more than one way to do this for an over-identified system and no way to do this for an under-identified system.
For the theory to be "scientific", the expression of the structure (3) must take a form such that the outcomes for the estimate of π are in some way limited. If the outcome of the estimation is not consistent with such limit(s), then the model has been falsified, i.e., Popper [3].
So far, the issues at stake seem straight-forward. Nevertheless, problems arise in terms of what it is about a proposed structure that is hypothesized by the theory; and, given this, just what limits does the hypothesis create for the esti- . For this, the analytic burden relates to showing that the sign pattern of β requires that at least some entries of the sign pattern of β −1 must have particular signs, independent of the magnitudes of the nonzero entries of β. In fact, much of the subsequent literature on what has become known as qualitative analysis treats the special case where γ = I, since if any entries of β −1 can be signed based upon the sign pattern initially hypothesized for β, it is reasonably easy, given whatever the sign pattern of γ is initially proposed to be, to see if any entries of π can be signed. The Monte Carlo approach identified below will readily detect any sign- under which a qualitative analysis would be successful was initiated by Lancaster [5] and is well-summarized in Hale, et al. [4]. None of this dispelled Samuelson's pessimism about the potential for a successful qualitative analysis, although Hale and Lady [6] is a significant exception. It is fair to say that such analyses are seldom performed and, if so, seldom found to be successful.
A recent literature, Buck and Lady [7] [8] and Lady and Buck [9], has shown that the concept of a qualitative analysis as discussed above was too restrictive.
Specifically, it was shown that although no sign in the reduced form could be signed, based upon the sign patterns of the structural arrays; nevertheless, there were (always) restrictions on what sign patterns that the reduced form could take. Given this, if the estimated reduced form took on a sign pattern forbidden by these restrictions, the proposed structural arrays are thereby falsified. As a simple example, for the case that γ = I and β −1 = π, the sign pattern of the estimated reduced form must be such that it is possible that βπ = I and πβ = I. Given this, the entries of the first row of the estimated reduced form π cannot, for example, all be the negative of the corresponding nonzeros proposed for the first column of β. This result holds even if no entries of the reduced form can be signed. If the estimated reduced form does not satisfy this restriction, the structural hypothesis has been falsified. Buck and Lady [7] [8] and Lady and Buck [9] describe a Monte Carlo technique for sampling quantitative realizations of the structural arrays and the corresponding reduced form that will identify such restrictions.
Buck and Lady [10] showed that the scope of restrictions due to a hypothesized qualitative structure was even broader than previously proposed. In particular, they showed that falsifiable restrictions on the reduced form can be imposed by the zero, and perhaps other, restrictions on the structure, independent of the (unrestricted) nonzero entries. As an example, consider the structure γ = I and, ? 0 0 ? ? ? 0 0 . 0 ? ? 0 0 0 ? ? In the above array, the entries marked "?" are nonzero, but may otherwise be of any sign and magnitude. The zero restrictions selected present an irreducible inference structure with only one cycle of inference. The Monte Carlo sampling and derivation of the reduced form for β −1 = π, found only 256 sign patterns out of the possible 65,536 sign patterns that a 4 × 4 array could take on, barring zeros. This result is due to the zero restrictions only and is independent of the nonzero signs and magnitudes of the other entries. Suppose a multistage least squares process were applied to this structure to estimate the nonzero entries. If any but the 256 reduced form sign patterns consistent with the structure were then found in the estimated reduced form, the resulting estimates of the nonzeros in the structure would be impossible, i.e., could not possibly generate a reduced form with the sign pattern found in the estimated reduced form. Consequently, any estimates of the structural nonzeros would be incorrect, since the zero restrictions themselves have been falsified.

One of the examples of falsification due to zero and other restrictions given in
Buck and Lady [10] is the Klein [11] Model I. This formulation has been a popular example for pedagogical purposes in econometrics, e.g., Berndt [2]. It has also been used as an example for qualitative analysis, e.g., Maybee and Weiner [12] and Lady [13]. Buck and Lady [10] showed that the sign pattern of the reduced form estimated by Goldberger [14] and used by Berndt [2] falsified the zero and other restrictions on the model's structural form, independent of the signs and magnitudes of the nonzero behavioral entries of the structural arrays.
However, this in itself is not a decisive finding. It turns out that Klein's Model 1 fails the Sargan [15] overidentification test for two out of three estimated equations.
In this paper we re-specify the original Klein Model I slightly, but enough to cause it to pass the overidentification tests and re-examine the resulting sign pattern on the theory that the model generated an impossible sign pattern because it was not properly identified in the first place. In the next section Klein An estimate of the reduced form for this model was provided by Goldberger [14]. The sign pattern of this estimated reduced form is given below, Significantly, Berndt [2], used the Goldberger [14] reduced form estimates for pedagogical purposes to demonstrate seven different multi-stage least squares methods to estimate the behavioral coefficients. Unknown to both Goldberger and Berndt, the zero and other restrictions on the structure are falsified by this sign pattern, independent of any signs or magnitudes assigned by whatever methods to the behavioral coefficients. In particular, Buck and Lady [10] reported that for the structure postulated as below, there did not exist nonzero signs and magnitudes that could be given to the structural entries marked "?" in the above that would result in the sign pattern of the estimated reduced form. As a result, all of the estimates given in Berndt [2] are impossible.

Is Klein Model I Properly Identified?
The Klein Model I is specified, for estimation purposes as follows (Goldberger 1964, 303-304

W c c E c E c Year u
Estimating the three behavioral equations using two-stage least squares yields the results shown in Table 1.
As noted above, this model has been shown to be impossible given the reduced form equations. One problem with the model, an issue that might be responsible for this unfortunate result, is that this model is not properly specified.
2 E Y W TX = − + Using the above notation, the amended model is expressed by, In writing out the expansions of the entries of π referred to in falsifying the KLM version of the model given in the Appendix, the above notation is used.
The resulting estimated KLM behavioral equations are presented in Table 2.
Does this solve the impossibility issue? We use the same Monte Carlo methodology reported on in Buck and Lady [10] to determine whether this sign pattern is possible. A version of the computer application that implements the Monte Carlo that we used is available at https://astro.temple.edu/~gmlady/RF_Finder/Finder_Page.htm which also contains the Stata programs and data used to estimate the models presented herein.
The reduced form estimated for the KLM model is given below in Table 3.
The Monte Carlo method takes quantitative samples of the structural arrays with the sign patterns given above, computes    1) The text files for β and γ are read into their corresponding sample arrays.
2) All entries in the sample-files are set equal to zero if the corresponding entries in the text-files are equal to zero, preserving the zero restrictions.
Having done this, we found that, ( ) ( ) ( ) π ). Thus, the signs of this triplet of entries in the estimated reduced form falsifies the model, i.e., are impossible. Further, this result is based entirely upon the accounting relationships (4) and (7) and is therefore independent of the signs and values of the behavioral coefficients. Thus, although our amendments to Klein's original specification solved the overidentification problem, the model is nevertheless falsified and is falsified independent of the signs and values of the behavioral coefficients.

Conclusions
The theme of this paper is that qualitative analysis and its potential to falsify a structural form based upon its sign pattern and the sign pattern of the estimated reduced form is a tool for model development as well as model dismissal. The  [10], and test the model for overidentification, which we found was present. Given this, we introduced some modest amendments to the original model, yielding our version, the KLM model.
These amendments resolved the overidentification problem, but further analysis revealed that, nevertheless, the KLM model was also falsified by the estimated In our opinion, when these restrictions are falsified by the estimated reduced form sign pattern, further development is appropriate and initiating the second stage of two stage least squares is not.