_{1}

^{*}

Factor analysis (FA) is a time-honored multivariate analysis procedure for exploring the factors underlying observed variables. In this paper, we propose a new algorithm for the generalized least squares (GLS) estimation in FA. In the algorithm, a majorization step and diagonal steps are alternately iterated until convergence is reached, where Kiers and ten Berge’s (1992) majorization technique is used for the former step, and the latter ones are formulated as minimizing simple quadratic functions of diagonal matrices. This procedure is named a majorizing-diagonal (MD) algorithm. In contrast to the existing gradient approaches, differential calculus is not used and only elmentary matrix computations are required in the MD algorithm. A simuation study shows that the proposed MD algorithm recovers parameters better than the existing algorithms.

Using

with

Here,

[

Three major appro ach es for the para mete r estimation are l east square s (LS), gene ra l ize d l east square s (GLS), and maxim um like li hood (ML) procedure s [

respectively, while

In all estimation procedure s, iterative algorithms are need ed for minimizing loss f unction

For all of the LS, GLS, and ML estimation , gradient algorithms have been dev e lop ed: t hose with the Fletcher- Powell and Newton-Raphson methods have been pro p ose d for the ML estimation [

As found in the above discussion, an in e qual ity - base d algorithm has not been dev e lop ed for the GLS estimation in which (4) is minimized over

The MD algorithm is not the first one with major ization in FA. In deed , the above EM algorithm [

The remaining parts of this paper are organized as follows: the MD algorithm is detailed in the next section, and it is illustrated with a real data set in Section 3. A simulation study for assessing the algorithm is reported in Section 4, which is followed by discussions.

We propose the MD algorithm for minimizing the GLS loss function (4) over the loadings in

where

and

This function is minimized over

Let us consider minimizing (7) over

Though the optimal

According to the formula, the update of

decreases the value of (8) with

with

In this section, we describe updating each of diagonal matrices

Here,

(11) is minimized for

for fixed

Next, we consider minimizing (7) over

with

for fixed

The results in the last two subsections show that the proposed MD algorithm can be listed as follows:

Step 1. Initialize

Step 2. Update

Step 3. Update

Step 4. Update

Step 5. Finish with

It should be noted in Step 2 that the update of

In Step 1, the initialization is performed using the principal component analysis of sample covariance matrix

In Step 5, we define the convergence as the decrease in the value of (7) or (4) from the previous round being less than

In this section, we illustrate the performance of the MD algorithm with a 190- person × 25- item data matrix, which was collect ed by the author and public ly avail able at http://bm.osaka-u.ac.jp/data/big5/. Th is data set con- tains the self - rating s of the person s (university students) for to what ex tent s they are character ize d by the personalities des crib e d by the 25 items . Ac cord ing to a theory in person a l ity psych ology [

We carried out the MD algorithm for the cor relation matrix with the number of factors m set to five.

Group | Item | Loading matrix | Y1_{p} | ||||
---|---|---|---|---|---|---|---|

Neurotic | Worry | 0.72 | −0.14 | −0.12 | 0.22 | 0.13 | 0.32 |

Sensitive | 0.74 | −0.06 | −0.04 | 0.01 | −0.08 | 0.38 | |

Pessimistic | 0.67 | −0.29 | −0.13 | −0.03 | 0.27 | 0.35 | |

Unrest | 0.60 | 0.06 | −0.15 | −0.08 | −0.34 | 0.33 | |

Careful | 0.62 | −0.09 | −0.16 | 0.10 | 0.32 | 0.32 | |

Extrovert | Sociable | −0.16 | 0.81 | 0.09 | 0.09 | 0.12 | 0.23 |

Talkative | 0.04 | 0.82 | −0.05 | −0.02 | −0.09 | 0.25 | |

Voluntary | −0.10 | 0.73 | 0.17 | 0.10 | 0.13 | 0.29 | |

Cheerful | −0.21 | 0.80 | 0.08 | 0.16 | 0.00 | 0.23 | |

Showy | 0.08 | 0.66 | 0.23 | −0.04 | −0.05 | 0.38 | |

Open | Creative | −0.09 | 0.03 | 0.85 | −0.04 | −0.07 | 0.23 |

Adventurous | −0.22 | 0.22 | 0.67 | −0.02 | −0.20 | 0.36 | |

Progressive | −0.19 | 0.24 | 0.64 | 0.08 | 0.10 | 0.40 | |

Flexible | −0.28 | 0.31 | 0.37 | 0.20 | 0.03 | 0.57 | |

Imaginative | 0.19 | 0.09 | 0.47 | 0.11 | −0.34 | 0.44 | |

Agree able | Mild | −0.15 | −0.12 | 0.15 | 0.63 | 0.02 | 0.38 |

Tenderhearted | 0.09 | 0.20 | 0.13 | 0.62 | −0.01 | 0.39 | |

Altruistic | 0.13 | 0.01 | 0.00 | 0.71 | 0.15 | 0.39 | |

Cooperative | 0.00 | 0.27 | −0.14 | 0.66 | 0.08 | 0.25 | |

Sympathetic | 0.11 | 0.12 | −0.04 | 0.73 | 0.21 | 0.32 | |

Conscious | Deliberate | 0.14 | −0.04 | 0.09 | 0.22 | 0.60 | 0.37 |

Reliable | −0.13 | 0.32 | 0.08 | 0.26 | 0.57 | 0.36 | |

Diligent | −0.04 | 0.05 | −0.07 | 0.15 | 0.77 | 0.29 | |

Systematic | 0.09 | −0.02 | −0.09 | 0.01 | 0.71 | 0.40 | |

Methodical | 0.21 | 0.02 | −0.19 | 0.04 | 0.74 | 0.31 |

As the result ing load ing matrix has rotational freedom, that is, the

A simulation study was performed in order to assess how well parameter matrices are recovered by the proposed MD algorithm and compare it with the existing algorithms for the GLS estimation in the goodness of the recovery. We first describe the procedure for synthesizing the data to be analyzed, which is followed by results.

An n-observations × p-variables data matrix

Here ,

Step 1. Draw

Step 2. Draw each loading in

Step 3. Draw each elements of

Step 4. Form

In Step 3 we have used a uniform distribution for

Let us express the true

can be used with

The statistics of AAD values over 2000 data sets are p resent ed in

Parameter | Loadings | Unique variances | ||||
---|---|---|---|---|---|---|

Algorithm | NR | GN | MD | NR | GN | MD |

Average | 0.0026 | 0.0020 | 0.0005 | 0.0030 | 0.0007 | 0.0000 |

50 percentile | 0.0000 | 0.0013 | 0.0005 | 0.0000 | 0.0001 | 0.0000 |

75 percentile | 0.0000 | 0.0031 | 0.0007 | 0.0000 | 0.0005 | 0.0000 |

95 percentile | 0.0050 | 0.0061 | 0.0011 | 0.0103 | 0.0035 | 0.0000 |

99 percentile | 0.0821 | 0.0095 | 0.0014 | 0.0857 | 0.0105 | 0.0001 |

Maximum | 0.2199 | 0.0203 | 0.0041 | 0.1406 | 0.0323 | 0.0013 |

s mall enough to be ignore d. That is, the proposed MD algorithm well recovered the true parameter values for all of the 2000 data sets. We can thus conclude that the MD algorithm is superior to the existing ones in the goodness of recovery.

We proposed the majorizing-diagonal (MD) algorithm for the GLS estimation in FA. In the algorithm, the loading matrix is reparameterized as the product of a column-orthonormal matrix and a diagonal one, and the former one is updated with Kiers and ten Berge’s [

One of the tasks remaining for the MD algorithm is to study its mathematical properties as have been done for the algorithms in the other estimation procedures. For example, it has been found that the EM algorithm for the ML estimation [