Efficient Generalized Inverse for Solving Simultaneous Linear Equations

Solving large scale system of Simultaneous Linear Equations (SLE) has been (and continue to be) a major challenging problem for many real-world engineering and science applications. Solving SLE with singular coefficient matrices arises from various engineering and sciences applications [1]-[6]. In this paper, efficient numerical procedures for finding the generalized (or pseudo) inverse of a general (square/rectangle, symmetrical/unsymmetrical, non-singular/singular) matrix and solving systems of Simultaneous Linear Equations (SLE) are formulated and explained. The developed procedures and its associated computer software (under MATLAB [7] computer environment) have been based on “special Cholesky factorization schemes” (for a singular matrix). Test matrices from different fields of applications have been chosen, tested and compared with other existing algorithms. The results of the numerical tests have indicated that the developed procedures are far more efficient than the existing algorithms.


Introduction
In scientific computing, most computational time is spent on solving system of Simultaneous Linear Equations (SLE) which can be represented in matrix notations as where A R × ∈ is a singular/non-singular matrix, and b is a given vector in n R .For practical engineering/ science applications, matrix A can be either sparse (for most cases), or dense (for some cases).For a non-singular coefficient matrix A, direct methods (Cholesky factorization, T LDL algorithm, LU decomposition, etc) or iterative methods (Conjugate Gradient algorithm, Bi-Conjugate Stabilization, GMRES, etc.) are used to solve Equation (1.1).If the coefficient matrix is singular or rectangular, the above mentioned direct and iterative methods cannot be used to solve Equation (1.1) and thus generalized inverse is needed to solve the unknown solution vector x in Equation (1.1).
Email: skadi002@odu.edu,dnguyen@odu.edu The generalized (or pseudo) inverse of a matrix is an extension of the ordinary/regular square (non-singular) matrix inverse, which can be applied to any matrix (such as singular, rectangular etc.).The generalized inverse has numerous important engineering and sciences applications.Over the past decades, generalized inverses of matrices and its applications have been investigated by many researchers [1]- [6].Generalized inverse is also known as "Moore-Penrose inverse" or "g-inverse" or "pseudo-inverse" etc.
In this paper we introduce an efficient (in terms of computational time and computer memory requirement) generalized inverse formulation to solve SLE with full or deficient rank of the coefficient matrix.The coefficient matrix can be singular/non-singular, symmetric/unsymmetric, or square/rectangular.Due to popular MATLAB software, which is widely accepted by researchers and educators worldwide, the developed code from this work is written in MATLAB language.
The rest of this paper is organized as follows.In Section 2, we discuss background of generalized inverse.In Section 3, we give a description of the algorithm.This section also describes the efficient generalized inverse formulation (which uses modified Cholesky factorization).In Section 4, we present comparison of numerical performances of the proposed algorithm with other existing algorithms.Extensive set of coefficient matrices (including rectangular, square, symmetrical, non-symmetrical, singular, non-singular matrices) obtained from well-established/popular websites [8] [9] were tested and the numerical performance in terms of timings, error norm were compared with other algorithms.Finally, conclusions are drawn in Section 5.

Singular Value Decomposition (SVD) and the Generalized Inverse
A general (square or rectangular) matrix where Σ 0, a diagonal matrix(does NOT have to be a square matrix) Σ 0, [ ] [ ] unitary matrices U and V = 1 (for real matrices) and Let A be a singular matrix of size m n × and let k be the rank of the matrix.Based on Equation (2.1), one has ; and Eigen Values of (or ) and Eigen-values of H A A × are the same.However, the Eigen-vectors of H A A × and Eigen-vectors of H A A × are "NOT" the same.Then, the generalized inverse A + of A is the n m × matrix and is given as where

Efficient Generalized Inverse Algorithms [1]-[3] [5] [6]
Moore-Penrose inverse can be computed using Singular Value Decomposition (SVD), Least Squares Method, QR factorizations, Finite Recursive Algorithm [2] [3], etc.In this work, our numerical algorithms have been based on: (a) The "special Cholesky factorization" (for symmetrical/singular coefficient matrix), and (b) The generalized inverse of a product of 2 matrices [6] and can be described in the following paragraphs.The Moore-Penrose inverse (or generalized inverse or pseudo inverse) of a m n × matrix K (not necessar- ily a square matrix) is the unique n m × matrix K + which satisfies the following four conditions: 1.General condition: , Normalized condition: ( ) Reverse normalized condition: ( ) with a square coefficient n n × matrix, and let the rank be less than the size of the ma-  M can be obtained by the regular/ standard Cholesky factorization, with the following modifications: a) When the diagonal term of the current th i row is very close to zero, then factorization of this dependent row is skipped.
b) When the current th i row is factorized, all previous rows 1, 2, , 1 k i = … − were used except those depen- dent row(s).
Consider the generalized inverse of a matrix product AB [1] [6] ( ) ( ) Let us consider regular inverse in Equation (3.5) in place of generalized inverse

AA A A AA A A A A A
Thus, Equation (3.7) becomes ( ) ( ) ( ) While MATLAB solution can be obtained by implying the generalized inverse G + [see Equation (3.9)] to be formed explicitly, our main idea is to solve SLE where b  is a known right-hand-side vector.

Numerical Performance of ODU Generalized Inverse Solver
Based on the detailed algorithms explained in Section 3, the numerical performance of our proposed procedures are evaluated in this section.The known RHS vector { } b can be random vector, or can be chosen such a way that the unknown solution vector { } { } We also compared the performance of our algorithm with the efficient algorithm described in [6] and also with MATLAB built-in function ( ) pinv [7] for computing the generalized inverse explicitly.We use MATLAB version 7.6.0.324 (R2008a) on Intel Core 2 CPU, 2.13 GHZ, 2GB RAM, Windows XP Professional SP3 for numerical comparisons.
Table 1 and Table 2 records the times (in seconds) taken by our proposed algorithm, the algorithm mentioned in [6] and MATLAB built-in function [7] ( ) pinv .For our convenience, we represent our algorithm with ( ) ODU ginverse − , algorithm in [6] with geninv and MATLAB built-in function with ( ) MATLAB pinv − .In addition, we have also presented the error norm for all the test matrices.

Conclusion
In this paper, various efficient algorithms for solving SLE with full rank, or rank deficient have been reviewed, proposed and tested.The developed numerical procedures can be applied to solve "general" SLE (in the form , where the coefficient matrix [ ] G could be square/rectangular, symmetrical/unsymmetrical, non-singular/singular).The users have option to choose either a direct solver or an iterative solver inside the generalized inverse to solve for SLE.Numerical results have shown that the proposed algorithms are highly efficient as compared to existing algorithms [6] (including the popular MATLAB built-in function

M 2 )
is the upper triangular (special) Cholesky factorized matrix and contains exactly n r − zero rows.Removing the zero rows from M, one obtains a r n × (upper, rectangular) matrix L′ .In this work, the upper triangular (special) Cholesky factorized matrix [ ]

7 )
From Equations (3.1)-(3.2) and Equation (3.6) one obtains, r is the rank of the matrix, then r n ≤ ).Let the size of the known right-hand-side vector b [6]lays the same role as matrix [ ]A in Equation (1.1)), then based on the theorem presented in[6], there exists a unique [ ]

Table 1 .
Computational times (in seconds) for symmetric rank-deficient test matrices with RHS Vector as linear combination of columns of coefficient matrix.

Table 2 .
Computational times (in seconds) for rectangular rank-deficient test matrices (Tall type: Rows >> Cols) with RHS Vector as linear combination of columns of coefficient matrix.