A Geometric Gaussian Kaczmarz Method for Large Scaled Consistent Linear Equations

This paper presents a geometric Gaussian Kaczmarz (GGK) method for solving the large-scaled consistent linear systems of equation. The GGK method improves the geometric probability randomized Kaczmarz method in [1] by introducing a new block set strategy and the iteration process. The GGK is proved to be of linear convergence. Several numerical examples show the efficiency and effectiveness of the GGK method.


Introduction
We are concerned with the approximation solution of large-scaled linear systems of equation of the form where m n A × ∈  is a real matrix, m b ∈  is a real vector and n x ∈  is an unknown vector to be determined. The Kaczmarz [2] method is a good candidate for solving large problems (1) due to its simplicity and good performance, which is applied in numerous fields, such as image reconstruction [3] [4] and digital signal processing [5]. The Kaczmarz method can be formulated as where ( ) In order to improve the influence on the convergence rate by diagonal scalings of the coefficient matrix in (1). Yang in [1] presented a geometric probability .
Generally, the block Kaczmarz methods [8] [9] select some rows from the coefficient matrix A to construct the block matrix and compute the Moore-Penrose pseudoinverse of the block at each iteration. However, the cost of computing the Moore-Penrose pseudoinverse of a matrix is so expensive, which impacts gravely the CPU time of the algorithm.
Gower and Richtrik proposed a Gaussian Kaczmarz (GK) method [10] whose iteration process is defined by where k ζ is a Gaussian vector with mean 0 m ∈  and the covariance matrix Here I denotes the identity matrix. The expected linear convergence rate was analyzed in the case that A is of full column rank.
The idea of the GK method is also used in [11] to avoid computing the pseudoinverses of submatrices. We will adapt this iteration process in our method.
In this paper, we improve the GPRK method in [1] and present a geometric Gaussian Kaczmarz (GGK) method. The GGK introduces a new block strategy and uses the iteration process of the GK method in (6). The block set strategy of GGK is more efficient than that in [1] because it can grasp as many as possible rows of the system matrix to participate in projection. This is illustrated in Section 3.
The rest of this paper is arranged as follows. A geometric Gaussian Kaczmarz Journal of Applied Mathematics and Physics algorithm is presented in Section 2. Its convergence is also proved. Section 3 shows several numerical examples for the proposed method and Section 4 draws some conclusions.

A Geometric Gaussian Kaczmarz Algorithm
This section describes a geometric Gaussian Kaczmarz (GGK) algorithm to compute the solution of (1). Algorithm 1 summarizes the GGK algorithm. Steps 2, 3 and 4 determine the block control sequence { } 0 k k τ ≥ , which is simpler than and different from that in [1], which determines the block control sequence by probability Proof. According to Algorithm 1, we have

Algorithm 1. A geometric Gaussian Kaczmarz algorithm (GGK). Journal of Applied Mathematics and Physics
Denote the projector ( ) denote the matrix whose columns orderly are constituted of all the vector

Numerical Experiments
In this section, we use Algorithm 1 for solving different types of consistent linear systems (1) and compare it with GPRK in [1].

Experiments with Sparse Matrix
This subsection considers the linear systems (1) with sparse matrices. These matrices include some flat ones in Table 1 and the tall ones (their transposes) in Table 2 from the University of Florida sparse matrix collection [12].   Table 1 and Table 2 Table 1 and Table 2 we see that the GGK method needs smaller IT and CPU than the GPRK method does in all cases. We can see whether system 1 is overdetermined or underdetermined. When the matrix is flat, the value of SU locates in the interval (2.70, 628.92), while for the case of the thin matrix, the value of SU ranges from 2.49 to 641.08. It is observed from Figure 1 and Figure 2 that the GGK method converges faster than the GPRK method.

Experiments with Dense Matrix
In this subsection, the test matrices are dense normally distributed random matrices including thin and flat matrices. The sizes of rows and columns of the selected matrices vary from 2000 to 30,000. Table 3 lists the test results of IT, CPU and SU for flat dense matrix with different size. Table 4 Table 1 and Table 2, we can     and Figure 2 are drawn from Figure 3 and Figure 4 that the GGK method converges much faster than the GPRK method.

Conclusion
We develop a geometric Gauss-Kaczmarz (GGK) algorithm for solving large-scale consistent linear systems and the convergence is proved for this algorithm. Numerical experiments show that the GGK algorithm has better efficiency and effectiveness than the GPRK algorithm. In our future work, we will focus on block Kaczmarz methods to solve ill-posed problems.